Jekyll2020-08-03T15:01:16+00:00http://kevinkle.in/feed.xmlBlogI like computer science and I find short but meaningful descriptions tough.
Kevin KleinOn Gifts2020-08-03T13:38:07+00:002020-08-03T13:38:07+00:00http://kevinkle.in/jekyll/update/2020/08/03/gifts<p>TL;DR: Spend thought not money. Think about your edge. Don’t buy beans for a hobby roaster.</p>
<p>I’ve often felt strongly about gifts made and received and always had some vague intuitions floating around in my head concerning distinctive properties.
Finally have I sought to distill them and make them more explicit.</p>
<p>Refelcting on past gift-finding processes, I came to realize that I would often feel dissatisfied when most of the value of
a gift would stem from money. I’d much rather have the value be derived from thought, concern and genuine interest about someone.
This topic concerns itself with the <em>kind of effort</em>. Within this kind of currency, efforts can still be misdirected. Pondering without
getting anywhere is likely a common phenomenon. What’s the purpose of spending the ‘right’ kind of effort
without ensuing reward? Hence, within the realm of spending thought on someone, I would attempt to direct them towards my own edge. In the
context of gifts my own edge seems to be:</p>
<ul>
<li>What I know a lot about in general, in particular at least as much as the receiver</li>
<li>What I know about the receiver that possibly not many other givers know</li>
</ul>
<p>I wish to emphaszie the former after having fallen victim to a fallacy more than once too often: gifting wine to a wine lover. Now there is a chance
that I’m a true and battle-proven wine afficionado, too. In that case, nevermind. In all other cases this person will have developed
strong preferences by sheer hours of exposure and attention to detail, quite literally acquired a taste. My blunt ‘guess’ seems way more likely to
miss than to hit. Clearly are there exceptions to this, e.g. me knowing about a detailed preference beforehand. But then, what exactly is the value
I am creating with a gift falling perfectly in line with something the receiver has had before, knows and could likely buy at the press of a button?</p>
<p>The latter seems fairly straight-forward to me. It seems like a classical pie expansion, win-win situation to me: the receiver can easily feel
appreciated, catered with attention and acknowledged as the snowflake that he or she is; I feel like doing a good job, creating value for my friend and
possibly eveing standing out from the crowd.</p>
<p>At the end of the day a good present is a present that makes everybody feel good about it and this is hard to capture in more explicit words. Yet, a priori,
I will now look for the following properties:</p>
<ul>
<li>The main currency <sup id="a0"><a href="#f0">[0]</a></sup> paid is thought and time, not money <sup id="a1"><a href="#f1">[1]</a></sup>.</li>
<li>The gift is unlikely to be gifted to the receiver by other givers.</li>
<li>The gift is unlikely to be gifted by me to a different receiver.</li>
</ul>
<p>The latter two properties seem like a reasonable heuristic for me having leveraged my edge <sup id="a2"><a href="#f2">[2]</a></sup>. That, in turn, seems like
a great precondition to creating true value. But it can of course still all go horribly wrong. :)</p>
<p><b id="f0">[0]</b> ‘Main’ is arguably very vague here. The heuristic of ‘what feels like the greatest cost?’ could be used here.</p>
<p><b id="f1">[1]</b> Except that… <a href="http://kevinkle.in/jekyll/update/2020/03/26/tim_update.html">time is money</a>.</p>
<p><b id="f2">[2]</b> Certainly not necessary, maybe not even sufficient conditions.</p>Kevin KleinTL;DR: Spend thought not money. Think about your edge. Don’t buy beans for a hobby roaster.Fig quiche2020-07-25T10:38:07+00:002020-07-25T10:38:07+00:00http://kevinkle.in/jekyll/update/2020/07/25/fig_quiche<h2 id="ingredients">Ingredients</h2>
<ul>
<li>300g shortcrust dough</li>
<li>large bunch of basil</li>
<li>~200g Soft, creamy goat cheese, e.g. Chavroux</li>
<li>~400g of figs (both fresh and dried are ok)</li>
<li>4-6 red onions</li>
<li>2 cloves of garlic</li>
<li>zest of one lemon</li>
<li>~3 eggs</li>
<li>~80ml cream <sup id="a0"><a href="#f0">[0]</a></sup></li>
<li>some lemon juice</li>
<li>some nutmeg</li>
<li>some sweet paprika powder</li>
<li>seasoning</li>
</ul>
<h2 id="method">Method</h2>
<p>Slice the red onions in rings and fry them under medium-high heat with both olive oil and
butter. Consider adding more oil to get to where you want to be. The onions having taken on
some color, add finely chopped garlic and maybe a bit of finely chopped seedless chili.
Once things are starting to dry up again, add a glug of red wine. Consider repeating
this step with some honey.</p>
<p>Coarsely chop the figs, goat cheese and basil. Mix with the onions.</p>
<p>Spread out the dough in the shape of your choice and cover it with your filling.</p>
<p>In a separate bowl, swirl up the eggs and cream. Pour the liquid over the filling and gently mix
things up with a spoon. Add the lemon zest, seasoning, some paprika, some nutmeg and a tad of lemon juice;
beware not to add too much liquid though. Excess of liquid and subsequent soaking of the dough is the ultimate
nemesis for quiche, imho. Depending on how sweet your figs are (or how bad your day has been) you might want
to consider drizzling some extra honey on top.</p>
<p>Place in the oven at around 220 degrees C until looking like a milion bucks.</p>
<h2 id="thoughts">Thoughts</h2>
<ul>
<li>Sprinkling some fresh thyme or rosemary into the red onions could have been interesting.</li>
<li>I find that this quiche goes particularly well with a spinach salad.</li>
<li>Having some nuts, e.g. pecans, in the quiche could also be interesting taste-wise. Yet,
I’m not sure if the crunchy texture would work well in the midst of creaminess.</li>
</ul>
<p><b id="f0">[0]</b> ‘Light or heavy cream?’ you may ask. Don’t you know the answer? :)</p>Kevin KleinIngredients 300g shortcrust dough large bunch of basil ~200g Soft, creamy goat cheese, e.g. Chavroux ~400g of figs (both fresh and dried are ok) 4-6 red onions 2 cloves of garlic zest of one lemon ~3 eggs ~80ml cream [0] some lemon juice some nutmeg some sweet paprika powder seasoningOrange pink eggs2020-06-06T19:38:07+00:002020-06-06T19:38:07+00:00http://kevinkle.in/jekyll/update/2020/06/06/eggs<p>I have a feeling that hard-boiled eggs are conventially viewed as unsexy. In spite of that, I gave pickling them a shot.</p>
<h2 id="ingredients">Ingredients</h2>
<ul>
<li>8 eggs</li>
<li>1 cup of white balsamic vinegar</li>
<li>1/2 cup of liquid from tinned beets</li>
<li>2 tbsp cardamon pods</li>
<li>2 tbsp cumin seeds</li>
<li>2 tbsp coriander seeds</li>
<li>2 tbsp pepper corns</li>
<li>1 tbsp fennel seeds</li>
<li>1 tbsp mustard seeds</li>
<li>a few cloves</li>
<li>a few bay leaves</li>
<li>2 cloves of garlic</li>
<li>2 tbsp curry powder</li>
<li>2 tbsp turmeric</li>
<li>2 tbsp sweet paprika powder</li>
<li>2 tbsp salt</li>
<li>3 tbsp sugar</li>
<li>1 tbsp garam masala</li>
</ul>
<h2 id="method">Method</h2>
<p>Boil the eggs as little as possible while making sure they’re actually boiled. If you found just the right spot, make
sure to put them into ice water as to prevent them continuing to cook themselves.</p>
<p>In the meantime toast your whole spices in a dry pan on medium heat until your kitchen smells like a market in a
far-flung exotic place. Eventually add the beet juice and ground spices to combine everything well. In particular,
make sure the sure is well disolved. Let cool for a bit.</p>
<p>Add your spice mixture and the vinegar into your mason jar.</p>
<p>Peel your hard-boiled eggs and add them, too.</p>
<p><img src="/imgs/eggs/spices.jpg" alt="spices" class="img-responsive" /></p>
<p>I let them pickle for roughly 7-10 days. They looked really cool - orange on the outside and pink on the inside.
They ended up being fairly acidic, partly overshadowing the taste of spices. I think it be cool if one could spare
some of the vinegar, although I’m not sure what this would imply for their shelf life.</p>
<p><img src="/imgs/eggs/outside.jpg" alt="outside" class="img-responsive" /></p>
<p><img src="/imgs/eggs/inside.jpg" alt="inside" class="img-responsive" /></p>Kevin KleinI have a feeling that hard-boiled eggs are conventially viewed as unsexy. In spite of that, I gave pickling them a shot.Caramelized Onion and Artichoke Ravioli2020-05-25T13:38:07+00:002020-05-25T13:38:07+00:00http://kevinkle.in/jekyll/update/2020/05/25/ravioli<h2 id="ingredients">Ingredients</h2>
<p>Dough</p>
<ul>
<li>300g of 00 flour</li>
<li>200g of ‘pasta flour’ (don’t actually know what that is)</li>
<li>5 eggs</li>
<li>some water</li>
</ul>
<p>Filling</p>
<ul>
<li>6 large onions</li>
<li>250g fresh ricotta (get the good, dry stuff)</li>
<li>~250g artichoke hearts</li>
<li>2 cloves of garlic</li>
<li>a glug of red wine</li>
<li>butter</li>
<li>tbsp sugar (optional)</li>
<li>thyme</li>
<li>zest of one lemon</li>
<li>some lemon juice</li>
<li>some nutmeg</li>
<li>some sweet paprika powder</li>
<li>seasoning</li>
</ul>
<h2 id="method">Method</h2>
<p>Use a standard approach to prepare the pasta dough, see for e.g.
<a href="https://leitesculinaria.com/40229/recipes-homemade-pasta-dough.html">this recipe</a>.</p>
<p>Slice the onions and start frying them in butter and oil. Eventually add finely chopped garlic, some thyme and
possibly some chili flakes. Add some red wine once your pan is low on moisture. Make sure not to be on too high
heat as you want the onions to fry and caramelize for a while - without starting to burn. Optionally add some sugar
at some point.</p>
<p>Let the onions cool off a bit and chop the artichoke hearts in the meantime.</p>
<p>Crumble the ricotta, grate some nutmeg and lemon zest. Combine all of the filling ingredients, add seasoning and
some lemon juice to your liking.</p>
<p>Gently fill your pasta and cook them.</p>
<p><img src="/imgs/ravioli/filled.jpg" alt="filled" class="img-responsive" /></p>
<p>Before placing them on a bed of arugula, I decided to gratinate the pasta with some cream, pecorino and a
beaten egg.</p>
<p><img src="/imgs/ravioli/grattin.jpg" alt="grattin" class="img-responsive" /></p>
<p>For reasons unknown to me the taste of the filling very much reminded of a ground beef filling.
I found the overall flavour to lack a hint of something I could not identify. Nevertheless I
enjoyed eating it a whole lot. In a way it tasted like classical comfort food pasta, yet surprising
me with every bite. There’s something really refreshing about the filling.</p>
<p>Many thanks to Julie.</p>Kevin KleinIngredients Dough 300g of 00 flour 200g of ‘pasta flour’ (don’t actually know what that is) 5 eggs some waterCreamy, Bittersweet, Sharp Pasta2020-05-10T07:38:07+00:002020-05-10T07:38:07+00:00http://kevinkle.in/jekyll/update/2020/05/10/chicory_gorgonzola<p>There’s just something great about caramelized chicory. While
I’ve often got there by covering it in honey and balsamic vinegar, I
was inclined to look for a similar effect with actual grapes. In
additional to the lovely interplay of bitterness and sweetness, there
is something light about the acidity of the grapes.</p>
<p>I figured that caramelized onions would only enhance this interplay
without distracting too much.</p>
<p>Combining Gorgonzola with pasta water can easily lead to a very creamy
cover for the pasta, as can be achieved with other types of
cheeses. Yet, the blue cheese combines the silky, creamy texture with
a sharpness in taste that pleasantly surprises.</p>
<h2 id="ingredients">Ingredients</h2>
<ul>
<li>500g of white grapes</li>
<li>4-5 big heads of chicory</li>
<li>300g of Gorgonzola</li>
<li>50g of walnuts</li>
<li>bunch of thyme</li>
<li>small bunch of parsley</li>
<li>some olive oil</li>
<li>some butter</li>
<li>shot of white wine</li>
<li>2 onions</li>
<li>3 cloves of garlic</li>
<li>some chili flakes</li>
<li>50g of Grana Padano</li>
<li>2 tbsp of honey</li>
<li>750 of rigatoni</li>
</ul>
<h2 id="method">Method</h2>
<p>Roughly chop the chicory in large chunks and halve the grapes. Put
them into a large container or onto an oven tray. The goal is for
everything to be in touch with the hot air, i.e. nothing should be
stacked. Toss both into the container of your choice, sprinkle some
thyme and drizzle the mixture with some olive oil. I’d recommend a
combination of grill and hot air circulation. A temperature of around
180 degrees Celsius for around half an hour worked fine for me. Mix
and stir from time to time and drizzle some honey halfway through.</p>
<p><img src="/imgs/chicory/chicory.jpg" alt="Chicory" class="img-responsive" /></p>
<p>Brown slices of onion in a mix of butter and olive oil. Add
finely chopped garlic, some chili flakes as well as some
walnuts. Eventually cook off some white wine.</p>
<p>While boiling the pasta with plenty of sea salt, make sure to slightly
under-cook the rigatoni and to save plenty of pasta water before
draining.</p>
<p>After draining the pasta, add them back into the pot used to boil
them. Incorporate the chicory-grape mixture as well as the onions. Add
the Gorgonzola and continually add pasta water while continuously
stirring. Keep on going until you reach the creaminess of your liking.</p>
<p>Finish off the mixture with some finely chopped parsley, Grana Padano,
freshly ground black pepper and sea salt.</p>
<p>Serve and hopefully enjoy. :)</p>Kevin KleinThere’s just something great about caramelized chicory. While I’ve often got there by covering it in honey and balsamic vinegar, I was inclined to look for a similar effect with actual grapes. In additional to the lovely interplay of bitterness and sweetness, there is something light about the acidity of the grapes.Markov Random Field Image Denoising2020-04-26T07:38:07+00:002020-04-26T07:38:07+00:00http://kevinkle.in/jekyll/update/2020/04/26/mrf<script type="text/javascript" async="" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?
config=TeX-AMS-MML_HTMLorMML"></script>
<p>This post describes the application of a Probabilistic Graphical Model for simple image denoising,
as suggested in <a href="https://www.microsoft.com/en-us/research/people/cmbishop/prml-book/">Bishop</a>’s
chapter 8.3.8. I will attempt to introduce some notions for general context without derivations.
Googling will lead to plenty of useful resources. On an introductory level, I enjoyed
<a href="https://cs.stanford.edu/~ermon/cs228/index.html">Daphne Koller’s class</a>.</p>
<h2 id="context-probabilistic-graphical-models">Context: Probabilistic Graphical Models</h2>
<p>Probabilistic Graphical Models are tools to model and express conditional
independencies among a set of random variables.</p>
<p>Why care about conditional independencies? Joint distributions can always be expressed without
conditional independencies. Yet, the latter allow for an expression of the joint distribution
requiring fewer parameters.</p>
<p>There are two major variants of Graphical Models. Some rely on directed graphs and use the notion of
‘d-separation’ for expressing conditional independencies. While there are quite a few flavors of
directed Graphical Models, Bayesian Networks are the ones I have encountered most. Another major variant
are undirected Graphical Models. A prominent representative of the latter group are ‘Markov Random Fields’,
or ‘MRF’ in short.</p>
<p>In Bayesian Networks, nodes of the graph, also called ‘factors’ have a direct probabilistic
interpretation: they represent conditional probabilities. In MRFs, the interpretation is less
straight-forward; more on that later. Some more information about the relation between directed and
undirected models:</p>
<ul>
<li>Any directed graph model can be turned into an undirected graph model via a
process called ‘moralization’. It consists of ‘marrying’ parent nodes. This process might remove
some conditional independencies.</li>
<li>In general, directed and undirected graph models can capture
different conditional independencies; see the next section for more detail.</li>
</ul>
<p>Remember than any joint distribution can be expressed without the help of conditional independencies.
This approach to leads to fully connected graph in Graphical Models.</p>
<h2 id="maps">Maps</h2>
<p>Having introduced the notions of both directed and undirected Graphical Models for representing joint distribution,
the question of how they relate arises naturally. This question is as interesting as (comparably) involved. I will limit myself
to presenting a fact indicating their distinct abilities.</p>
<p>Let’s define three types of ‘maps’ and illustrate how they interact with one another. Assume the distribution comes with random
variables <script type="math/tex">A, B, C</script>.</p>
<ul>
<li>D(ependency)-map: <script type="math/tex">A \not\perp B \| C</script> in distribution <script type="math/tex">\Rightarrow</script> <script type="math/tex">A \not\perp B \| C</script> in graph</li>
<li>I(ndepdence)-map: <script type="math/tex">A \not\perp B \| C</script> in distribution <script type="math/tex">\Leftarrow</script> <script type="math/tex">A \not\perp B \| C</script> in graph</li>
<li>P(erfect) map: <script type="math/tex">A \not\perp B \| C</script> in distribution <script type="math/tex">\Leftrightarrow</script> <script type="math/tex">A \not\perp B \| C</script> in graph</li>
</ul>
<p>We previously made the statement that directed and undirected graphs can, a priori, capture
different conditional independencies. This can be formalized by looking into the set of perfect maps
of directed and undirected graphs respectively. Let’s call the set of all perfects maps
<script type="math/tex">\mathcal{P}</script>, the set of all perfect maps captured by directed graphs <script type="math/tex">\mathcal{P}_{DG}</script> and
<script type="math/tex">\mathcal{P}_{UG}</script> analogously. The relationship of said sets can be illustrated as follows:</p>
<p><img src="/imgs/mrf/map_sets.png" alt="map sets" class="img-responsive" /></p>
<h2 id="context-markov-random-fields">Context: Markov Random Fields</h2>
<p>In MRFs, the joint distribution can be expressed as a product of potential functions over all maximal cliques of the graph. This implies the principal lever
of the technique: for per-node operations, dependencies are reduced from a ‘times all other nodes’ to a ‘times all other nodes of maximal cliques this node belongs to’. <sup id="a0"><a href="#f0">[0]</a></sup>
Let’s (hopefully) clarify this with a motivational example.</p>
<p>Given the set of Random Variables <script type="math/tex">X_1, ..., X_N</script>. Assume we are given the information that the distribution can be modeled via MRFs with a chain graph and potential functions <script type="math/tex">\psi_{i, i+1}</script>. Note that maximal cliques are any pairs of neighbors.</p>
<p><img src="/imgs/mrf/chain_graph.png" alt="chain graph" class="img-responsive" /></p>
<p>The goal is to answer the query: For a given <script type="math/tex">x_i'</script>, what is <script type="math/tex">p(X_i=x_i')</script>?</p>
<p>A naive approach would go as follows: evaluate and marginalize the joint distribution. In other words:
\[p(x_i’) = \sum_{x_1’}\sum_{x_2’} \dots \sum_{x_{i-1}’}\sum_{x_{i+1}’} \dots \sum_{X_N’} p(x_1’, x_2’, \dots, x_{i-1}’, x_i’, x_{i+1}’, \dots, x_{N}’)\]
Assuming that every random variable is discrete and can take on <script type="math/tex">O(k)</script> many values, this approach leads to <script type="math/tex">O(k^N)</script> evaluations of the joint distribution.</p>
<p>Leveraging the conditional independencies express through the MRF we can do much better. While we still rely on marginalization, we can do it in a more efficient way by rearranging sum terms.</p>
<p>\[p(x_i’) = \frac{1}{Z} \sum_{x_1’}\sum_{x_2’} \psi_{1,2}(x_1’, x_2’) \sum_{x_3’}\psi_{2,3}(x_2’, x_3’) \sum_{x_4’}\psi_{3,4}(x_3’, x_4’) \dots \sum_{x_N’} \psi_{N-1, N}(x_{N-1}’, X_N’)\]
with <script type="math/tex">Z</script> being a normalization term.</p>
<p>Observe that this computation comes with merely <script type="math/tex">O(K^2N)</script> evaluations of the potential functions.</p>
<p>Now this may leave quite a few questions open, such as: What’s the deal with the potential function? What’s the
transition of ‘evaluations of the potential function’ to number of parameters? While those are central concerns,
my ambition is confined to giving you a glimpse of the situation and, in the best case, tease you to explore the
great literature, which will provide answers, yourself.</p>
<h2 id="application-image-denoising">Application: Image denoising</h2>
<h3 id="givens">Givens</h3>
<ul>
<li>An observed black-and white image <script type="math/tex">Y \in \{-1, 1\}^{N \times M}</script></li>
</ul>
<h3 id="assumptions">Assumptions</h3>
<ul>
<li>The observed image <script type="math/tex">Y</script> is the result of an underlying ‘true’ image <script type="math/tex">X \in \{-1, 1\}^{N \times M}</script> suffering noise pollution.</li>
<li>
<p>The noise flips every pixel i.i.d. with fixed probability <script type="math/tex">p</script>. I.e.</p>
<p><script type="math/tex">Y_{ij} = X_{ij} + \epsilon_{ij}</script> with
<script type="math/tex">% <![CDATA[
\epsilon_{ij} =
\begin{cases}
-2 X_{ij},& \text{ with probability } p\\
0 &\text{ with probability } 1 - p
\end{cases} %]]></script></p>
</li>
<li>The neighborhood of a pixel explains all of its variance. In other words, a pixel is independent
of all non-adjacent pixel conditioned on all adjacent pixels.</li>
</ul>
<h3 id="goal">Goal</h3>
<ul>
<li>Intuition: Reconstruct the underlying image <script type="math/tex">X_{ij}</script> by investigating the noisy observation <script type="math/tex">Y_{ij}</script>.</li>
<li>Explicit: Generate <script type="math/tex">\hat{Y}</script> with <script type="math/tex">\sum_{ij} \|(Y_{ij} - \hat{Y_{ij}})\|</script> a small as possible.</li>
</ul>
<h3 id="model">Model</h3>
<ul>
<li>
<p>Graph:</p>
<p><img src="/imgs/mrf/mrf_graph.png" alt="probability updates" class="img-responsive" /></p>
<p>We observe that there are two types of edges in the graph:</p>
<ul>
<li>edges between <script type="math/tex">X</script> and <script type="math/tex">Y</script>, e.g. <script type="math/tex">X_{ij}</script> and <script type="math/tex">Y_{ij}</script></li>
<li>edges within <script type="math/tex">X</script> and <script type="math/tex">y</script> respectively, e.g. <script type="math/tex">X_{ij}</script> and <script type="math/tex">X_{i+1,j}</script></li>
</ul>
<p>The former result from the nature of the noise process: an observed pixel is directly linked to the underlying pixel. The latter result from the assumption that the immediate neighborhood of a pixel is its <a href="https://en.wikipedia.org/wiki/Markov_blanket">Markov blanket</a>. In other words, knowing the neighbors of the pixel, no other pixel will add any information about the pixel under consideration.</p>
<p>As compared to the complete graph, we clearly have <em>very few</em> edges in this graph. Moreover, we only have a constant <a href="https://en.wikipedia.org/wiki/Degree_(graph_theory)">degree</a>. This will turn out very handy.</p>
<p>The transition from both kinds of edges allows for a very natural transition to two kinds of maximal cliques:</p>
<ul>
<li>cliques between <script type="math/tex">X</script> and <script type="math/tex">Y</script>, e.g. <script type="math/tex">X_{ij}</script> and <script type="math/tex">Y_{ij}</script></li>
<li>cliques within <script type="math/tex">X</script> and <script type="math/tex">Y</script> respectively, e.g. <script type="math/tex">X_{ij}</script> and <script type="math/tex">X_{i+1,j}</script></li>
</ul>
</li>
<li>
<p>Potential functions:</p>
<p>The potential functions directly follow from the previous observations of types of maximal cliques. Per pixel <script type="math/tex">(i, j)</script>, we can express both potential functions:</p>
<ul>
<li>\(\Psi_a (X_{ij}, Y_{ij}) = -\eta_a X_{ij} Y_{ij}\)</li>
<li>\(\Psi_b (X_{ij}, X_{i’j’}) = -\eta_b X_{ij} X_{i’j’} \text{ for } (i’, j’) \in \text{ neighborhood}(i,j)\)</li>
</ul>
</li>
<li>
<p>Energy function:</p>
<p>While the potential functions are defined per pixel, the energy function is defined over the image, with
<script type="math/tex">(i', j')</script> stemming from the neighborhood of <script type="math/tex">(i ,j)</script> once again.
\(E(X,Y)) = \sum_{i, j} \Psi_a (X_{ij}, Y_{ij}) + \sum_{(i, j), (i’, j’)} \Psi_b (X_{ij}, X_{i’j’}) \)</p>
</li>
<li>
<p>Optimization algorithm:</p>
<p>Having established the energy function, the next step is to define a way of minimizing/maximizing it.
Bishop suggests a very simple greedy algorithm. Its upside is that it enhances in every step, converges and is local. Its downside is that it doesn’t
necessarily converge to the global optimum.</p>
<ul>
<li>initialize <script type="math/tex">\hat{Y}=X</script></li>
<li>repeat until convergence:
<ul>
<li>choose <script type="math/tex">i, j</script></li>
<li>\(y_{i, j} := argmax_{\hat{y}_{i, j}} E(X, Y)\)</li>
</ul>
</li>
</ul>
<p>Note that the locality of this algorithm is thanks to <script type="math/tex">y_{i,j}</script>’s effect on <script type="math/tex">E</script> is limited to its neighborhood. In other words, the rest of the
graph can be disregarded. This locality allows for convenient parallelization. Let’s finish off with the looking into experiment results.</p>
</li>
<li>
<p>Experiment:
Starting with a black and white Mondrian, I introduced noise by flipping pixels with a probability of .2. The algorithm was run with a non-adaptive stopping criterion of
updating every pixel 10 times. Again, the ‘true image’ corresponds to <script type="math/tex">Y</script>, the ‘noisy image’ to <script type="math/tex">X</script> and the ‘denoised image’ to <script type="math/tex">\hat{Y}</script>.</p>
</li>
</ul>
<p><img src="/imgs/mrf/true.png" alt="chain graph" class="img-responsive" />
<img src="/imgs/mrf/noisy.png" alt="chain graph" class="img-responsive" />
<img src="/imgs/mrf/denoised.png" alt="chain graph" class="img-responsive" /></p>
<p>You can find corresponding code in my <a href="https://github.com/kklein/mrf-denoiser">repo</a>.</p>
<p><b id="f0">[0]</b> I <em>did</em> try to rephrase this sentence. <a href="#a0">↩</a></p>Kevin KleinPlaylist Chatbot2020-03-27T14:00:00+00:002020-03-27T14:00:00+00:00http://kevinkle.in/jekyll/update/2020/03/27/playlist-chat-bot<p>Countless times have I found myself in the situation of receiving a telegram message of following
kind:</p>
<p><img src="/imgs/playlist-chat-bot/telegram-message.png" alt="telegram message" class="img-responsive" /></p>
<p>On the one hand I find this immensely exciting:</p>
<ul>
<li>A friend takes time to share something with me.</li>
<li>I take pleasure in exploring music per se.</li>
<li>Both conjuncted, there is a possibility to bond over music.</li>
</ul>
<p>Hence there is a great amount of potential!</p>
<p>Yet, on the other hand, it’s a bit of a mess. As seemingly few people do, I use Google Play
Music/Youtube Music over spotify (it’s a long story). Spotify links are sometimes rendered
nicely by Telegram as to indicate artist/song/album information. Sometimes not. Depending on that,
the process of adding a friend’s suggestion to a playlist of mine looks as follows:</p>
<ul>
<li>? Follow spotify link (leave telegram), retrieve song information.</li>
<li>Open Google Play Music.</li>
<li>Query song information.</li>
<li>Add a search result to playlist.</li>
<li>Switch back to Telegram to thank dear friend. :)</li>
</ul>
<p>My desire was to keep all of the upside while getting rid of all context changes. This could be
achieved by a chatbot. The desired behavior:</p>
<ul>
<li>Friend messages chatbot directly with song information in plain text.</li>
<li>Chatbot queries Google Play Music for songs.</li>
<li>Chatbot adds highest-ranking result to my playlist.</li>
<li>Chatbot confirms addition to friend.</li>
</ul>
<p><img src="/imgs/playlist-chat-bot/bot-message.png" alt="bot message" class="img-responsive" /></p>
<p>Implementing such a bot came with fairly low effort as I found both the <a href="https://python-telegram-bot.org/">python telegram bot
api</a> as well as the <a href="https://github.com/simon-weber/gmusicapi">gmusicapi</a> very user-friendly. Deploying the bot locally with a
polling mechanism has also proven to be relatively straightforward.</p>
<p>Remote deployment seemed a bit trickier at first, for Google play music authentication is typically
handled via oauth browser authentication. Thanks to the help of <a href="https://www.simonmweber.com/">Simon
Weber</a> I was able to make it run on Google Cloud Functions. This has
actually been a surprisingly seamless and pleasant experience. Details regarding the authentication
can be found in the <a href="https://github.com/kklein/gmusic-bot">project repo</a>.</p>Kevin KleinCountless times have I found myself in the situation of receiving a telegram message of following kind:Update: Time Is Money2020-03-26T14:38:07+00:002020-03-26T14:38:07+00:00http://kevinkle.in/jekyll/update/2020/03/26/tim_update<p>A while back, I wrote about <a href="http://kevinkle.in/jekyll/update/2018/08/03/tim.html">Time Is Money</a>.</p>
<p>Back then I ran into several technical problems with modern browsers’ <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/idle">idle api</a>.</p>
<p>Circumventing these with a somewhat brute approach made it possible to get the extension to run more smoothly. Besides still being able to build it from source, it is now available in the <a href="https://chrome.google.com/webstore/detail/timeismoney/pdhagfofcpkciigdpbmjniblellpbjjk">chrome extension store</a>.</p>
<h2 id="current-behaviour">Current behaviour</h2>
<p>In the extension options, a user can specify an <em>hourly</em> wage as well as a set of <em>undesirable websites</em>.</p>
<p>Whenever the user has an active tab on one of those websites, a counter will appear in the top left corner of the browser window. This is periodically updated to reflect the ‘amount spent’ on undesirable websites throughout the current week.</p>
<p>The count should only consider the most recently active tab when working with several browser windows. Yet, this is all <em>a bit</em> messy. There are plenty of edge cases: multiple browser windows, multiple browsers with different profiles, several OS windows open, OS hibernation, laptop lid closes and many more.</p>
<p>I attempted to capture some of those aspects explicitly. For all others, a simple timeout mechanism should bound the errors: after a user-defined timeout of inaction, the count will no longer be increased until there is further action.</p>
<p>Please reach out in case you bump into unintended behaviors or have suggestions for improvements!</p>
<p>For some more technical context, please see the <a href="https://github.com/kklein/timeismoney">project repo</a>.</p>Kevin KleinA while back, I wrote about Time Is Money.Invitations2020-03-02T07:38:07+00:002020-03-02T07:38:07+00:00http://kevinkle.in/jekyll/update/2020/03/02/invitations<script type="text/javascript" async="" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?
config=TeX-AMS-MML_HTMLorMML"></script>
<h2 id="scenario-and-goal">Scenario and goal</h2>
<p>You are the host of an event for which there are \(n\) candidates to be invited and you possess sufficient resources to host all candidates. Let the nature of the event be such that it seems personally attached to you, yet also fairly polarizing in its experience. In other words:</p>
<ul>
<li>some invitees will have a desire to decline</li>
<li>invitees with a desire to decline will feel bad about declining</li>
<li>some non-invitees will have a desire to attend</li>
<li>non-invitees with a desire to attend will feel bad about not being able to attend</li>
</ul>
<p>Moreover, one might weigh the ambitions wrt non-desiring invitees and desiring non-invitees differently. For instance, one might consider the discomfort of someone having
to decline, and thereby expressing his/her lack of appreciation, worse than the missed opportunity of someone not attending who would’ve desired to.</p>
<p>Concretely, this gives rise to the goal of <em>minimizing</em> the following quantities:</p>
<ul>
<li>non-desiring invitees</li>
<li>desiring non-invitees</li>
</ul>
<p>Thus, recalling the option to weigh those terms differently, we obtain the following loss function:
\[\mathcal{L} := \alpha \cdot \text{#non-desiring invitees} + (1 - \alpha) \cdot \text{#desiring non-invitees}\]
with \(\alpha \in [0, 1]\).</p>
<p>If we had perfect a priori knowledge about candidates desires and would either invite according to it or its negation, our loss function would evaluate to \(0\) or \(n\) respectively. As those represent ‘best’ and ‘worst’ cases, we can set our expectation of algorithm performance to be within the range \([0, n]\).</p>
<h2 id="assumptions">Assumptions</h2>
<ul>
<li>Candidate desires to attend the event are i.i.d. with \(\Pr[\text{desire}] = q\).</li>
<li>\(q\) is unknown.</li>
<li>Invitation candidates respond to the invitation according to their desire.</li>
</ul>
<h2 id="algorithms-and-their-performances">Algorithms and their performances</h2>
<p>Let’s start off with some naive deterministic algorithms before we delve into randomized approaches.</p>
<h1 id="naive-1-invite-all-candidates">Naive 1: Invite all candidates</h1>
<p>\begin{align}
\mathbb{E}[\mathcal{L}] &= \alpha \mathbb{E}[\text{#non-desiring candidates}]\\
&= \alpha n (1-q)
\end{align}</p>
<h1 id="naive-2-invite-nobody">Naive 2: Invite nobody</h1>
<p>\begin{align}
\mathbb{E}[\mathcal{L}] &= (1 - \alpha) \mathbb{E}[\text{#desiring candidates}] \\
&= (1 - \alpha)n q
\end{align}</p>
<h1 id="randomized-1-invite-candidates-iid-with-prtextinvitep">Randomized 1: Invite candidates i.i.d. with \(\Pr[\text{invite}]=p\)</h1>
<p>\begin{align}
\mathbb{E}[\mathcal{L}] &= \alpha \mathbb{E}[\text{#non-desiring candidates}] \Pr[\text{invite}] + (1 - \alpha) \mathbb{E}[\text{#desiring candidates}] \Pr[\text{no invite}] \\
&= \alpha n (1 - q) p + (1 - \alpha)n q (1 - p)
\end{align}</p>
<p>Naturally, one might wonder what shape the optimal \(p\) takes on, as a function of \(\alpha\) and \(q\).</p>
<p>\begin{align}
\hat{p} :&= argmin_{p}\ \mathcal{L} \\
&= argmin_{p}\ \alpha n (1 - q) p + (1 - \alpha)n q (1 - p) \\
&= argmin_{p}\ p (\alpha (1 - q) + (1 - \alpha)q(-1)) \\
&= argmin_{p}\ p (\alpha - q)
\end{align}</p>
<p>Recalling that \(p\) is a probability, we know it is bounded by \([0, 1] \). Hence there are three cases:</p>
<ul>
<li>\(\hat{p} := 0\) if \(\alpha > q\)</li>
<li>\(\hat{p} := 1\) if \(\alpha < q\)</li>
<li>\(p\) is irrelevant if \(q = \alpha\)</li>
</ul>
<p>What does this tell us?
In the first case, we would like to go with the naive ‘invite nobody’ approach whereas in the second case, we would like to go with the ‘invite everybody’ approach. Intuitively this might come as a slight surprise - why aren’t there any cases in which the optimal \(p\) lies <em>inbetween</em> \(0\) and \(1\), after all? I think the route towards insight as to why this doesn’t happen is to emphasize that all candidates are drawn i.i.d. with \(p\).</p>
<p>However, as \(q\) is assumed to be unknown, we cannot evaluate said condition. In other words, depending on \(\alpha\) and \(q\), one of Naive 1 and Naive 2 will perform well, the other badly. This randomized approach will lie somewhere in-between. Allowing for an <em>adaptive</em> approach, i.e. letting \(p\) vary over time, we could ‘push’ \(p\) closer to either \(0\) or \(1\), whichever is preferable. Such a method would naturally end up with a loss between that of the fixed starting probability \(p\) and the approached ‘invite all’ or ‘invite nobody’ approach. For instance, if asking everybody is the optimal choice, the relative ordering of expected losses looks as follows:</p>
<p><img src="/imgs/losses.png" alt="losses" class="img-responsive" /></p>
<p>The red arrows indicate the direction in which we’d like to shift the randomized approach by making it adaptive.</p>
<p>Note that in order to devise an adaptive strategy, we need to consecutively sample feedback. Hence candidates invitations are no longer i.i.d. but rather invited one after another, each possibly influencing all of the remaining candidates.</p>
<h1 id="randomized-2-update-invitation-propabilities-p_i">Randomized 2: Update invitation propabilities \(p_i\)</h1>
<p>Given a \(p\) such that \(p_1 = p\) and \(\delta > 0\).</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>for i = 1 to n:
invite candidate i with probability p_i
if candidate i was invited:
if candidate i accepts:
p_{i+1} := p_i + delta
else:
p_{i+1} := p_i - delta
</code></pre></div></div>
<p>We assume that \(delta\) is small enough such that \([0, 1] \subset [p_1 - n \delta; p_1 + n \delta]\). If not, one could simply cut \(p_i\) off at 0 and 1 respectively.</p>
<p>In order to analyze the expected loss of this approach, it turns out useful to ask another question first: How many candidates will be invited with this approach?</p>
<p>The iterative process of updating \(p_i\) can be thought of as a random walk on the following Markov chain:</p>
<p><img src="/imgs/invitation_mc.png" alt="probability updates" class="img-responsive" /></p>
<p>By definition, the likelihood of candidate \(i\) receiving an invitation depends on the state we find ourselves in in timestep \(i\). Referring to a state and its associated value \(p_i\) by \(S_i\), our question of the total amount of invitees translates to:</p>
<p>\[\mathbb{E}[\text{#invitees}] = \mathbb{E}[\sum_{i=1}^n S_i]\]</p>
<p>Before addressing the evaluation of this quantity, we first derive a helpful notion: the expected state in timestep \(i\).
\begin{align}
\mathbb{E}[S_i] &= \mathbb{E}[p_1 + \delta \cdot \text{#‘right’ transitions so far} - \delta \cdot \text{#‘left’ transitions so far}]\\
&= p_1 + \delta \cdot \mathbb{E}[\text{#‘right’ transitions so far}] - \delta \cdot \mathbb{E}[\text{#‘left’ transitions so far}] \text{ (L.O.E.)}\\
&= p_1 + \delta (i-1)q - \delta(i-1)(1-q)
\end{align}</p>
<p>Coming back to our underlying question, we obtain:
\begin{align}
\mathbb{E}[\text{#invitees}] &= \mathbb{E}[\sum_{i=1}^n S_i] \\
&= \sum_{i=1}^n \mathbb{E}[S_i] \text{ (L.O.E.)} \\
&= \sum_{i=1}^n p_1 + \delta (i-1)q - \delta(i-1)(1-q) \\
&= n p_1 + \delta \frac{n(n-1)}{2}(2q - 1) \\
&= n (p_1 + \delta \frac{n-1}{2}(2q - 1))
\end{align}</p>
<p>For the sake of compactness, let us define \(r := p_1 + \delta \frac{n-1}{2}(2q - 1)\).
Plugging this into our loss function, we learn that:</p>
<p>\begin{align}
\mathbb{E}[\mathcal{L}] &= \alpha \mathbb{E}[\text{#non-desiring candidates}] \Pr[\text{invite}] + (1 - \alpha) \mathbb{E}[\text{#desiring candidates}] \Pr[\text{no invite}] \\
&= \alpha n (1 - q) r + (1 - \alpha) n q (1 - r)
\end{align}</p>
<p>… so did we really improve on the first randomized, non-adaptive approach, with fixed \(p\)? A closer examination of \(r\) yields the insight. We see that \(r < p_1\) if \(q < \frac{1}{2}\) and \(r > p_1\) if \(q > \frac{1}{2}\). In other words, this approach is an enhancement over the former if either is satisfied:</p>
<ul>
<li>\(\alpha > q\) and \(\frac{1}{2} > q\)</li>
<li>\(\alpha < q\) and \(\frac{1}{2} < q\).</li>
</ul>
<p>On the one hand, it might seem like a disappointment that it is not always an improvement over the randomized approach with fixed probability. On the other hand, we now see (more clearly) why such hopes were unfounded: Our update rule is independent of \(\alpha\). Yet, \(\alpha\) determines the loss function. An alternative approach, paving the way towards strict superiority and leveraging \(\alpha\) in the update step seems very feasible, albeit possibly less simple and clean. :)</p>
<p>Furthermore, note that the dependence of \(r\) on \(n\) implies that given the satisfaction of either condition, the loss of this approach converges towards the loss of optimal fixed \(p\) with increasing number of candidates, \(n\).</p>
<h2 id="closing-remarks">Closing remarks</h2>
<ul>
<li>It seems to me that what we are doing in the update step can be seen as implicitly maintaining and leveraging a counter on the binary outcomes of desire preferences. This is a flavor of explore-exploit.</li>
<li>The <em>addition</em> of \(\delta\)s in the update step is a design choice. Multiplication seems like a sensible approach, too.</li>
<li>\(\delta\) is a hyperparameter.</li>
<li>Many thanks to Tim and Daan.</li>
</ul>Kevin KleinNot buying tableware2020-01-23T17:38:07+00:002020-01-23T17:38:07+00:00http://kevinkle.in/jekyll/update/2020/01/23/tableware<p>While mindlessly strolling around the leftover 18th century fortifications of a coastal Moroccan town I came across some streets with vendors of apparently local tableware. I found said tableware to be really beautiful. Sadly, the pure state of admiration only lasted briefly. The latter was soon accompanied and then progressively consumed by the concern of what to buy for whom. It got me quite excited (in a twisted way) and thinking hard about how my own tableware could be complemented, how others <em>could</em> use <em>any</em> tableware. Suddenly I wondered: Why am I thinking of buying things?</p>
<p>I didn’t start off with the intention of buying something. I just bumped into some objects and found those to be beautiful - in a pure, naive, maybe innocent way. How had this turned into a tedious search for a function mapping object-person pairs to likelihoods of appreciation? <sup id="a0"><a href="#f0">[0]</a></sup></p>
<p>Well, clearly, the displayal of the tableware was not in the context of a museum or gallery but rather in that of a commercial exchange. In other words the framing turned my refreshing and uncynical enthusiasm for an object - a rare sight - into a minor headache.</p>
<p>Note that the root of the headache was to <em>find a use</em> for the object. It seems to me that there are two obvious approaches to dampen this wound:</p>
<ul>
<li>Buy the object without a use, i.e. basically throw it away in expectation, over short or long term.</li>
<li>Don’t purchase and thereby don’t look for a use.</li>
</ul>
<p>The former option isn’t too compelling to me because of the principle of wastefulness as well as some kind of disrespect towards the craftsman. I figure: it’d be really cool if I could decouple the initial, innate reaction to the object from the reaction to the context of the object, i.e. go with the latter option. In other words: appreciate it without thinking of having to buy it. While this might not be the most natural thing to do, it certainly seems to be in the realms of the feasible. It’s certainly not jedi-level mind control. Yet, as everyone who’s been to Morocco knows, vendors can be rather pushy…</p>
<p>Assuming I succeed in the decoupling of appreciation and drive for purchase, I suspect I would feel bad raising the hopes of vendors by meticulously inspecting their objects (which I would actually really like to do) without ever buying something. This, again, could be attempted to be mitigated by meditating it away wrt the effect on myself. Hence, I might be able to not feel bad, but the vendor would still have had his hopes crushed. Note that this is a natural and important process of learning, of updating priors based on evidence. My point being here that my datapoint would not <em>improve</em> the vendors estimator. It would incur a loss on him (having his hopes crushed) without enhancing the accuracy of his judgement as I believe that such a way of behaving is far from representative of normal people. I’d be a costly outlier that his loss function is not prepared to identify and appropriately disregard. Additionally, <em>maybe</em> more practically speaking, I would waste a lot of their time with perfect knowledge thereof.</p>
<p>My proposed ideal world: in a PSA I let all the vendors know about my dilemma and offer them a fee to inspect their objects, much like in a museum or gallery. This give me the ease of mind to inspect to full satisfaction without bothering second thoughts and hopefully makes the craftsmen and vendors feel appropriately appreciated for their work and possession.</p>
<p><b id="f0">[0]</b> Note that this assumes that gifts or possessions in general are not per se of positive value. If they were, just buying anything for anyone would be an obvious solution to the problem. I don’t believe in this, though. I’m under the impression that gifts can, under certain circumstances, cause the recipient more trouble than gain <sup id="a1"><a href="#f1">[1]</a></sup>. Therefore I try to assess this beforehand, which as you can see, can cause some trouble on my side… <a href="#a0">↩</a></p>
<p><b id="f1">[1]</b> E.g.: ‘What should I do with this item for which I have no use? I can’t just throw it away, can I? Should I just put it in the basement? Is this establishing a precedent for keeping <em>de facto trash</em> in my basement? Will this leading to panic attacks once I’m moving again? Is this raising the bar for changing flats? Is this raising the bar for changing cities? Is this preventing me to accept $great_job? Is this making me unhappy?’ <sup id="a2"><a href="#f2">[2]</a></sup> <a href="#a1">↩</a></p>
<p><b id="f2">[2]</b> II wonder how, on a scale from 1 to 10, 1 being a standup routine and 10 being a eulogy, you think how serious I am. <sup id="a3"><a href="#a3">[3]</a></sup> <a href="#a2">↩</a></p>
<p><b id="f3">[3]</b> I’m only through one of David Foster Wallace’s books and completely fell in love with footnotes. Monkey see, monkey do. <a href="#a3">↩</a></p>Kevin KleinWhile mindlessly strolling around the leftover 18th century fortifications of a coastal Moroccan town I came across some streets with vendors of apparently local tableware. I found said tableware to be really beautiful. Sadly, the pure state of admiration only lasted briefly. The latter was soon accompanied and then progressively consumed by the concern of what to buy for whom. It got me quite excited (in a twisted way) and thinking hard about how my own tableware could be complemented, how others could use any tableware. Suddenly I wondered: Why am I thinking of buying things?