Jekyll2024-02-09T09:42:44+00:00http://www.csun.io/feed.xmlCameron SunThe personal website of Cameron Sun.
Cameron SunExpected Wins - Blaming Fantasy Sports Failure on Bad Luck2024-02-09T00:00:00+00:002024-02-09T00:00:00+00:00http://www.csun.io/2024/02/09/expected-wins-fantasy<p>I’m in a fantasy soccer league with some guys who spend more time complaining about the game than actually strategizing about how to play it. The crux of the problem is that results are heavily influenced by matchups. Even if you outscore all but one of the other players in your league, you’ll still lose if that higher scoring player happens to be your opponent for the week. On the flip side, you can score terribly during a week and still record a win purely because you’re lucky enough to play against a worse team. This element of chance makes for an easy scapegoat when you lose. When you win, though, your opponents will use it to bring up various stats that “prove” that you didn’t deserve to.</p>
<p>Two of the most frequently referenced stats are the total amount of points scored by (for) or against a team over the course of a season. It’s often the case that the league leader has a much lower Points Against total than anyone else. This tends to be interpreted as a sign that the league is rigged in favor of easier matchups for a certain person. However, I’d argue that Points Against is a misleading metric. The highest performing teams in the league only have to play against lower performers, naturally leading to fewer points scored against them. A low Points Against total can just be a sign that a team is performing well relative to others, not the cause of the high performance itself.</p>
<p>Even Points For is not necessarily a good metric because it fails to take consistency into account. Consider a 10 game season with two teams. Team A scores 90 points every week, while Team B scores 1000 points in one game and then 0 the rest of the season. Team A will finish with a 9-1 record, but Team B will finish with a higher Points For total. The owner of Team B will try to claim that they’ve been robbed, and that their greater Points For total proves that they’re the better team. But the stats don’t show that Team A had far superior consistency, which is ultimately what the game rewards in the long run.</p>
<p>I’ve come up with a simple metric that accounts for these shortcomings. I call it Expected Wins (xW). The formula is as follows:</p>
\[xW\ =\sum _{n\ =\ 1}^{\#\ weeks\ in\ season} \ \frac{\#\ of\ teams\ outscored\ in\ week\ n}{\#\ of\ other\ teams\ in\ league}\]
<p>Expected Wins is just the average number of wins you’d expect to have if you faced a random opponent every week of the season. Like Points For and Points Against, this is based on your actual performances throughout the season. Unlike those metrics, it also takes your consistency into account by limiting the amount of influence any given gameweek can have on the metric as a whole. A large gap between your expected and actual wins over the course of a season implies that you’ve gotten very (un)lucky. I know from googling around that I’m not the first person to think of this. However, I figured it was worth discussing because it doesn’t seem to be displayed by any major fantasy sports sites.</p>
<p>Here’s a plot of xW over time for my current season.</p>
<p><img src="/images/expected_wins/plot.png" alt="" /></p>
<p>As you can see, the only problem with this metric is that it (accurately) shows that my team is the best. It’ll be tough to sell that to the other guys in my league - especially given that I’m currently in fourth place.</p>Cameron SunI’m in a fantasy soccer league with some guys who spend more time complaining about the game than actually strategizing about how to play it. The crux of the problem is that results are heavily influenced by matchups. Even if you outscore all but one of the other players in your league, you’ll still lose if that higher scoring player happens to be your opponent for the week. On the flip side, you can score terribly during a week and still record a win purely because you’re lucky enough to play against a worse team. This element of chance makes for an easy scapegoat when you lose. When you win, though, your opponents will use it to bring up various stats that “prove” that you didn’t deserve to.Ellipsis Split V22023-03-02T00:00:00+00:002023-03-02T00:00:00+00:00http://www.csun.io/2023/03/02/ellipsis-split-v2<p>Here’s my latest keyboard - the successor to the original <a href="/2020/06/14/ellipsis-split-buildlog.html">ellipsis split</a> and <a href="/2018/07/16/custom-mechanical-keyboard.html">ellipsis</a> keyboards I’ve made over the past few years.</p>
<p>I went for a much more minimalist design this time around to cut down on costs and make it more feasible to build multiple copies of the keyboard. The keyboard is constructed from a brushed + anodized aluminum plate on top of a pcb, and sourcing those two parts from China cost me about $150 for 5 units. Pretty good for such a low order quantity.</p>
<p>The layout itself is intentionally almost identical to a standard keyboard. The toggles on the left half of the keyboard switch between mac/windows layouts and enable/disable “space mode” - binding all thumb cluster keys to spacebar. This is useful for playing games with default keybinds (as usually space is on the right half of the keyboard,) or for making it so that strangers can easily use the keyboard without trying to figure out which key is space.</p>
<p>You can find the files for the pcb + plate <a href="https://github.com/csun/ellipsis_split_files">here</a> and the firmware <a href="https://github.com/csun/qmk_firmware/tree/ellipsis_split_v2">here</a> on the <code class="language-plaintext highlighter-rouge">ellipsis_split_v2</code> branch. Enjoy the pictures below!</p>
<p><img src="/images/ellipsis_split_v2/ellipsis_dark_both.jpg" alt="" />
<img src="/images/ellipsis_split_v2/ellipsis_dark_left.jpg" alt="" />
<img src="/images/ellipsis_split_v2/ellipsis_light_both.jpg" alt="" />
<img src="/images/ellipsis_split_v2/ellipsis_light_left.jpg" alt="" />
<img src="/images/ellipsis_split_v2/ellipsis_light_right.jpg" alt="" />
<img src="/images/ellipsis_split_v2/ellipsis_light_switches.jpg" alt="" /></p>Cameron SunPictures and some info about my 3rd custom split mechanical keyboard design and build.Literal Raytracing - Visualizing Light’s Path Through Space2022-11-28T00:00:00+00:002022-11-28T00:00:00+00:00http://www.csun.io/2022/11/28/literal-raytracer<p>In a sense, we can’t actually see light.</p>
<p>Bear with me here.</p>
<p>Imagine turning on a flashlight and pointing it at the night sky. An uncountable number of photons stream out of the bulb every second, but you can’t actually observe them directly. It may be hard to even tell that the light is on. Place your hand in the path of the beam, though, and you’ll see a very clear bright spot where the light strikes your skin.</p>
<p>When we perceive light, we’re actually experiencing photons striking us directly in the eye. Conversely, we <em>cannot</em> perceive photons that do not hit us in the eye. That’s why you can’t see a flashlight beam that’s shining away from you, but you can see its light bouncing off of other objects (like your hand.) In particularly dusty / foggy environments you may actually be able to see a light shafts, but only because there are particles in the air scattering photons into your eye.</p>
<p>But what if we could change that? What would it look like if we could see light’s path as it flies through the air? To find out, I built a computer simulation. You can see some of its output below.</p>
<p class="img-caption">
<video width="100%" controls="" loop="" autoplay="" muted="">
<source src="/videos/literal_raytracer/sunburst.mp4" type="video/mp4" />
Your browser does not support this video.
</video>
A white light shining down from behind a golden sphere.
</p>
<p class="img-caption">
<video width="100%" controls="" loop="" autoplay="" muted="">
<source src="/videos/literal_raytracer/ladder.mp4" type="video/mp4" />
Your browser does not support this video.
</video>
A white light bouncing between green and blue metallic walls.
</p>
<p>In this post, I’ll give an overview of how this simulation works, as well as some demonstrations of how different material properties affect the way objects interact with light.</p>
<h1 id="implementation-overview">Implementation Overview</h1>
<p>As you may have guessed from the title, the simulation is essentially just a <a href="https://en.wikipedia.org/wiki/Ray_tracing_(graphics)">raytracer</a> built using <a href="https://unity.com/">Unity</a> (disclaimer: I work there.) However, instead of sampling the rays that strike each pixel of the camera sensor like a traditional raytracer, this simulation renders the entire path of each cast ray (literally “tracing” the ray. Get it?)</p>
<p>The main simulation loop can be broken up into three main steps: raycasting, tracing, and rendering.</p>
<p>In the raycasting step, we go through a queue of rays and cast them into the scene. This step is used to figure out where each ray hits an object and terminates (if at all). For any rays that do manage to strike an object, new rays are created and enqueued for future casting. The new ray parameters are based on the angle of reflection and various material properties (I’ll get into specifics about this later). Finally, if there’s space left in the queue, it is topped up with new rays randomly shot out of light sources in the scene.</p>
<p>The raycasting step happens on the CPU, and uses the Unity physics system for actually calculating the ray hitpoints. That means that reflections are based on object colliders rather than their actual mesh geometry. In addition to performance issues, doing this step with the physics engine is problematic because mesh colliders are limited to 256 triangles by Unity. I got around this by only using primitive (sphere, box, etc.) colliders and highly simplified mesh colliders. If I were to do this for real, though, I’d look into GPU raytracing support and stay far away from the physics system.</p>
<p>In the tracing step, info about all the cast rays (start, end, color) is passed to a GPU compute shader. This shader’s job is to update two different screen-sized textures - one for the average color and one for the total brightness of all rays that have crossed each pixel. The shader runs on a per-pixel basis, checking if each ray cast during the simulation passes through the given pixel. If it does, that ray’s color is used to update the pixel’s average color and total brightness values. This shader also checks the depth of each point along the ray against the <a href="https://en.wikipedia.org/wiki/Z-buffering">scene depth buffer</a> at that position, so that occlusion of rays by scene objects works as expected.</p>
<p>Finally, rendering the image is as simple as taking the output of the tracing step, scaling all the color values based on their total brightness values, then drawing to the screen. I wrote a shader to find the brightest pixel in the image and scale all brightnesses logarithmically proportional to that max. In hindsight there are probably many superior ways of doing this. I think this process of going from high to low dynamic range is generally called tonemapping, but I didn’t know that at the time.</p>
<p>Here’s an example of a simulation running slowly, casting a single ray at a time. I’ve overlayed a faint image of what the scene looks like with a “normal” camera so that you can see what all the rays are bouncing off of. Note that the “normal” view has had an extra light added to it so that objects are visible - in the actual simulation scene, it’s just a light in the bottom left, along with two cubes and a monkey head.</p>
<p class="img-caption">
<video width="100%" controls="" loop="" autoplay="" muted="">
<source src="/videos/literal_raytracer/monkey_single.mp4" type="video/mp4" />
Your browser does not support this video.
</video>
</p>
<p>And here’s the same simulation sped up and casting multiple rays per simulation step:</p>
<p class="img-caption">
<video width="100%" controls="" loop="" autoplay="" muted="">
<source src="/videos/literal_raytracer/monkey_multi.mp4" type="video/mp4" />
Your browser does not support this video.
</video>
</p>
<p>One thing worth noting is the way that overlapping rays are blended. For most of this simulation, I tried to stick to physical precedent, but there isn’t really one here. Are light rays solid? Translucent? Do they have a thickness? The method I ended up using is essentially a per-pixel weighted average of all the rays that have passed through it, but it would be interesting to try some different blending algorithms to see how things are affected.</p>
<h1 id="material-properties">Material Properties</h1>
<p>For me, the most interesting part about this project was learning about how different materials reflect light. In school, you’re taught that <a href="https://farside.ph.utexas.edu/teaching/316/lectures/node127.html#:~:text=The%20law%20of%20reflection%20states,the%20normal%20to%20the%20mirror.">“the angle of incidence equals the angle of reflection”</a> - but there’s much more that goes into creating realistic computer graphics.</p>
<h2 id="color">Color</h2>
<p>The simplest and most obvious property is material color. Physical materials reflect different wavelengths (colors) of light at different rates. To simulate this, our virtual materials define a “base color”. When a ray strikes an object, the color of the reflected ray is set to the base color multiplied by the color of the incident ray. The logic here changes a bit for non-metals, but we’ll get to that in a later section.</p>
<h2 id="roughness">Roughness</h2>
<p>In the real world, even seemingly flat surfaces have tons of microscopic scratches and ridges. These little peaks and valleys change the angle of incidence of any given ray, meaning that a reflected ray may end up shooting in a direction that’s not exactly mirrored about the overall surface normal.</p>
<p class="img-medium"><img src="/images/literal_raytracer/roughness_pbr.png" alt="" /></p>
<p class="img-caption">Source - <a href="https://substance3d.adobe.com/tutorials/courses/the-pbr-guide-part-1">Adobe PBR Guide</a></p>
<p>In the above image, you can see an example of this effect. Though the surface of the sphere is macroscopically smooth, the microscopic ridges cause the incoming rays (orange arrows) to be reflected in somewhat unpredictable directions (green arrows). This is why the reflections of the clouds and scenery on the red sphere are a bit murky rather than being clearly defined. The surface roughness adds a bit of “fuzziness” to the sphere’s reflections, blurring sharp lines and features.</p>
<p class="img-caption">
<video width="100%" controls="" loop="" autoplay="" muted="">
<source src="/videos/literal_raytracer/high_roughness.mp4" type="video/mp4" />
Your browser does not support this video.
</video>
</p>
<p>Here you can see how a rough, golden, metallic plate is modelled in the simulation. During the raycasting step, some gaussian noise is applied to the surface normal at each ray contact point. This causes the reflected rays to scatter randomly - much like the reflected rays shown in the previous diagram.</p>
<p><img src="/images/literal_raytracer/high_roughness_observers.png" alt="" /></p>
<p>If we superimpose two hypothetical observers onto the simulation, we can reason about how each of they each perceive the reflection from the rough plate. Observer A is situated right along the path of the “ideal” reflection (where the angle of incidence equals the angle of reflection). As such, they experience a bright and wide golden reflection from the plate. Observer B is positioned at a lower angle, on the fringes where fewer rays are scattered. Therefore, Observer B will see a much dimmer reflection than Observer A.</p>
<p class="img-caption">
<video width="100%" controls="" loop="" autoplay="" muted="">
<source src="/videos/literal_raytracer/low_roughness.mp4" type="video/mp4" />
Your browser does not support this video.
</video>
</p>
<p>Here’s the same simulation, but with a perfectly polished blue metal plate. Because there’s no surface roughness at all, each ray is reflected perfectly such that its angle equals the angle of incidence. This results in a very tight, focused beam of reflected light.</p>
<p><img src="/images/literal_raytracer/low_roughness_observers.png" alt="" /></p>
<p>When we overlay Observers A and B in the exact same spots as before, we can see that something different has happened. Now, Observer A is getting the full force of the reflection. Their eyes are effectively absorbing as much energy as if they were staring directly into the light. This is probably very unpleasant.</p>
<p>Observer B, on the other hand, cannot see any reflection at all. Because the reflections off of the perfectly smooth material are so focused, it means that you need to be in a much more specific position to see any reflected light at all. As a sanity check, here is how a “normal” 3D renderer displays these materials from each Observer vantage point:</p>
<p class="img-medium"><img src="/images/literal_raytracer/roughness_observer_fpv.png" alt="" /></p>
<p>We can see that everything is exactly as predicted from our simulation - wide, fuzzy reflections from the rough plate that are dimmer from Observer B’s position, and a hyper focused reflection from the smooth plate that disappears entirely for Observer B.</p>
<h2 id="metalness">Metalness</h2>
<p>Note that up until this point, we’ve only been talking about metallic materials. That’s because there’s a fundamental difference between the way that metals and non-metals interact with light.</p>
<p>To understand why, imagine turning on a flashlight and putting a piece of aluminum foil over the bulb. No light will escape. Do the same with a sheet of wax paper, though, and you’ll be able to see light shining through - even though the two materials are roughly the same thickness. This is because metals fully absorb all the light that they don’t reflect, whereas nonmetals can allow light pass through. Of course, most objects we see are not as thin as a piece of paper, but this principle still holds. Thick, non-metallic objects allow light to penetrate the surface at a microscopic level. This seemingly minor effect is primarily responsible for the visible differences between metals and non-metals.</p>
<p class="img-medium"><img src="/images/literal_raytracer/metallic_pbr.png" alt="" /></p>
<p class="img-caption">Source - <a href="https://substance3d.adobe.com/tutorials/courses/the-pbr-guide-part-1">Adobe PBR Guide</a></p>
<p>The above diagram portrays light striking a non-metal. The rays labeled “Specular” are the ones being reflected directly off of the surface. Specular reflections behave in the way we described in the roughness section above, and exist in both metals and non-metals</p>
<p>The main difference between metals and non-metals lies in the rays labeled “Diffuse”. Notice how the incident ray in the diagram actually penetrates the surface of the material. When a ray penetrates like this, it bounces around on the material’s constituent particles until it’s either fully absorbed as heat, or until it manages to bounce back out of the surface of the material and into the environment. If a ray manages to exit the material’s surface, we can assume that it has bounced around so much that it’s effectively travelling in a new, random direction.</p>
<p class="img-caption">
<video width="100%" controls="" loop="" autoplay="" muted="">
<source src="/videos/literal_raytracer/metalness.mp4" type="video/mp4" />
Your browser does not support this video.
</video>
</p>
<p>Here’s a simulation of lights striking a pink plastic plate (left), and a white metal plate (right.) The first thing you’ll probably notice is that the light striking the pink plastic is being sprayed haphazardly all around, whereas the light striking the metal is sent in a very specific direction. This is because the plastic is non-metallic. Incident light is able to penetrate its surface, bounce around, and diffuse in completely random directions. The metal, on the other hand, absorbs all light that isn’t immediately reflected, and is therefore only able to produce a specular reflection. The implication for observers is that the pink rays from the plastic can be seen from all angles, and are a relatively even brightness regardless of viewer position. On the other hand, metal varies wildly in brightness based on how close an observer’s eyes are to the center of the specular reflection.</p>
<p>It’s worth pointing out that non-metals still create specular reflections. In fact, you can see this in the simulation. The slightly brighter orange streak in the bottom left is the plastic’s specular reflection, and an observer looking directly at those rays would see a bright spot (just as we saw in the metallic examples.) This is why some materials (polished marble, glossy plastic) can still look “shiny” while still being distinctly non-metallic. If you can reduce a material’s surface roughness enough, it’ll develop the same sharp, specular highlights that polished metals have.</p>
<h2 id="fresnel-effect">Fresnel Effect</h2>
<p>The final thing I want to show is a really cool phenomenon called the Fresnel Effect.</p>
<p>In short, the Fresnel Effect describes the tendency for surfaces to get more reflective as you get closer to parallel to them. You can see this effect when looking at a wide variety of surfaces, but I’ll use the surface of a lake as an example here. When standing on the shore and staring at the water, the closer to your feet you look, the closer to perpendicular your viewing angle will be to the surface, and the less reflective the water will appear. Staring out farther from shore, the opposite is true, and the water will appear almost mirror-like. You can see this in the image below. The sand beneath the water can be seen near the bottom of the image, but not farther away, where the reflections of the mountains and sky can be seen instead.</p>
<p class="img-medium"><img src="/images/literal_raytracer/lake_tahoe.jpg" alt="" /></p>
<p>The Fresnel Effect is described mathematically by the <a href="https://en.wikipedia.org/wiki/Fresnel_equations">Fresnel Equations</a>, which calculate the ratio of light that is reflected when striking a surface. The formula returns higher reflectance values for incident rays that are closer to parallel with the struck surface, hence the Fresnel Effect. In the case of the simulation, we compute this reflectance value (or rather, <a href="https://en.wikipedia.org/wiki/Schlick%27s_approximation">Schlick’s Approximation</a>) every time a ray strikes an object. From there, we cast a new ray for the reflected component and scale its intensity based on the computed value. This ray constitutes the specular component of the reflection.</p>
<p>By subtracting the reflectance value from one, we get the amount of incident light that is transmitted into the struck surface. As discussed in the previous section, this component is entirely absorbed by metallic materials. For non-metallic materials, we scatter a second ray in a random direction and scale its intensity by the transmittance ratio times some constant positive factor that’s less than one (to simulate loss of energy from absorbtion). This constitutes the diffuse component of the reflection.</p>
<p class="img-caption">
<video width="100%" controls="" loop="" autoplay="" muted="">
<source src="/videos/literal_raytracer/fresnel.mp4" type="video/mp4" />
Your browser does not support this video.
</video>
</p>
<p>Above, you can see three lights striking three identical planes made of a blue non-metallic material. You can see that each object reflects both diffuse (blue) and specular (white) rays, but the ratio of blue to white is different for each object. The closer the object gets to being parallel with the light source, the more rays are reflected specularly rather than entering the material and being scattered as part of the diffuse reflection. That’s the Fresnel Effect in action.</p>
<h1 id="closing-thoughts">Closing Thoughts</h1>
<p>This was a really fun project to work on. I had some familiarity with material properties before, but I think that seeing how light bounces around “behind the scenes” really helped deepen my understanding. Now I often find myself looking around at various surfaces, trying to follow the path of light through the environment.</p>
<p>If you’d like to mess around with the simulation yourself, you can get the Unity project source <a href="https://github.com/csun/literal_raytracer">on github.</a> It might be a fun project to try converting it to use the GPU more effectively, so that rays could be cast more quickly. It’d be cool to see these sorts of simulations in realtime with moving objects and cameras, but that would require quite a big performance upgrade.</p>
<p>I’d also like to see if someone can use this to get good visualizations of more complex scenes. When trying to set up scenes with more than 2-3 light bounces, I found the results were often way too visually cluttered to take anything meaningful from them. Maybe the blending algorithm needs to be reconsidered, or some layer of realtime interactivity needs to be added (eg. controls to show only light bounce 2 or bounces from object X.) I also have full texture sampling support set up for materials, but in practice couldn’t find any demo scenes I wanted to show that made use of that.</p>
<p>Have fun!</p>Cameron SunWhen we perceive light, we're actually experiencing photons striking us directly in the eye. Conversely, we cannot perceive photons that do not hit us in the eye. But what if we could change that? What would it look like if we could see light's path as it flies through the air?How I See Numbers2022-03-03T00:00:00+00:002022-03-03T00:00:00+00:00http://www.csun.io/2022/03/03/how-i-see-numbers<p><em>*Please see the associated <a href="https://news.ycombinator.com/item?id=30539292">Hacker News comments</a> for some interesting discussion related to this article.*</em></p>
<p>When doing any sort of mental arithmetic or visualization of quantities, I picture numbers as shapes. In my head, operations like addition and subtraction manifest themselves as physical interaction between these shapes, and are complete with sounds and tactile feelings.</p>
<p>It only struck me recently that this might not be how most people experience numbers. In fact, this document is my first ever attempt to communicate this information with someone else. It was surprisingly difficult for me to represent the shapes and sensations given that I can conjure them so easily within my own head, but hopefully the following combination of drawings and words gets the point across.</p>
<p>One, like its written form, is just a line. It’s solid and indivisible. Two is a pair of ones, stuck to one another by a sort of invisible force. This force is quite strong. If overcome, the component ones can be peeled cleanly apart - like separating two slices of Kraft singles that have been stuck together (weird analogy, but that’s exactly how it feels.)</p>
<p><img src="/images/how_i_see_numbers/numbers_1.jpg" alt="" class="img-small" /></p>
<p>All of the even numbers are very neat and square. They stack nicely to combine into one another, and separate just as easily into pieces of two, like Lego bricks. The numbers that add to ten have an especially strong attractive force between them. They nestle into one another and don’t want to be pulled apart. The smaller of the two numbers always stacks on top of the larger, never the other way around.</p>
<p><img src="/images/how_i_see_numbers/numbers_2.jpg" alt="" /></p>
<p>In contrast with the pristine, cookie-cutter evens, the odds are much more organic. Three is quite round, like a one draped lazily over a two. Seven is the opposite, with a concavity that the three is designed to slot into. Just like with the evens, the odds want to latch together to join into a ten.</p>
<p><img src="/images/how_i_see_numbers/numbers_3.jpg" alt="" /></p>
<p>Nine is a monolithic block with a slot gouged out of it, for receiving a one. It functions like a hook, peeling off a one from any number it’s added to. This process leaves behind a nice, complete ten and one other digit - the reduced form of the other addend.</p>
<p><img src="/images/how_i_see_numbers/numbers_4.jpg" alt="" /></p>
<p>Numbers tends to vary their form contextually. Five, for example, can either be a two balanced delicately atop a three, or a four with an additional hook-like protrusion - much like the top half of a nine. The form any given five takes on depends on where it’s come from or what I need to do with it. In the presence of a seven, the five morphs into its two plus three form. The bottom half squishes into the top of the seven and the two on top just slides right off. In the presence of another five, both assume the hooked form and interlock to form a perfect ten (though this is the weakest feeling bond of all the numbers adding to ten.)</p>
<p><img src="/images/how_i_see_numbers/numbers_5.jpg" alt="" /></p>
<p>As orders of magnitude change, the feeling of each numerical interaction stays the same, but the scale of the effect grows or shrinks accordingly. Adding 0.25 and 0.25 feels like the sharp click of glove compartment latch, whereas 25 plus 25 feels like slamming a trunk closed - the same mechanisms at play, but much more forceful and solid at larger scales.</p>
<p>Beyond the first ten natural numbers, some have unique forms, but most do not. Instead, I’ll usually just break numbers up into their component base ten digits and do any sort of interaction with other numbers on a digit-by-digit basis.</p>
<p>And that’s all that I’ve got - hopefully this has been interesting! I find it funny that people can experience such different things when thinking about something so simple, and it scares me a bit knowing that I’ll never fully understand anything other than how it happens in my own head.</p>
<p>On the other hand, maybe you’ve been nodding along this whole time going “yup, that’s exactly how I see numbers too!” and none of this is a surprise to you.</p>
<p>Actually, that scares me a hell of a lot more. I prefer to believe that we all see things a little differently :)</p>Cameron SunWhen doing any sort of mental arithmetic or visualization of quantities, I picture numbers as shapes. In my head, operations like addition and subtraction manifest themselves as physical interaction between these shapes, and are complete with sounds and tactile feelings.Generative Art: Pen Plotting an Old Family Photo2021-12-29T00:00:00+00:002021-12-29T00:00:00+00:00http://www.csun.io/2021/12/29/plotting-old-pictures<p class="img-caption"><img src="/images/plotter_portrait/unframed_full.jpg" alt="" />
A portrait of my dad, aunt, and grandfather. The piece is a heavily processed and vectorized version of an old photo of the three of them, plotted on watercolor paper.</p>
<p class="img-caption"><img src="/images/plotter_portrait/texture_closeup_2.jpg" alt="" class="img-medium" />
A closeup of some of the texture generated by the pen plotter. Plotting gives a lot of subtle variation that you don’t get from normal printers.</p>
<p>While looking through old family photos, I really fell in love with one of my dad and aunt as kids, standing with my grandfather in front of their station wagon. I wanted a large physical version for my wall, but didn’t want to just blow up a low-resolution JPEG and print it out. So, I figured I’d try my hand at artistically recreating the photo and plotting it with a <a href="https://en.wikipedia.org/wiki/Plotter">pen plotter</a>.</p>
<p>In this post, I document all of the steps I took to turn an old picture into a piece of plotter art. I assume basic familiarity with command line tools, but otherwise hope most reasonably computer-literate people will be able to follow along and generate some art of their own.</p>
<h1 id="image-vectorization">Image Vectorization</h1>
<p>Plotters are essentially robot line-drawing machines. Given a list of lines and curves, a plotter will physically move a pen around a piece of paper and draw each one. The one big advantage of plotting vs. printing is that images can be plotted at any size with no loss in resolution. Whereas normal printed images will start to show their individual pixels when blown up to a large size, plotted images will keep their smooth, continuous lines without becoming blocky or jagged.</p>
<p>Cameras do not output lines and curves directly - they output pixels (or exposed film which is converted to pixels 50 years later.) As such, normal image files cannot be drawn by plotters. Instead, any image that you want to plot using a pen plotter needs to first be converted from pixels to lines and curves. This conversion process is called vectorization, as we’re converting from a <a href="https://en.wikipedia.org/wiki/Raster_graphics">raster</a> (pixel-based) image format to a <a href="https://en.wikipedia.org/wiki/Vector_graphics">vector</a> (shape-based) one. Here’s the original raster image that I want to vectorize:</p>
<p><img src="/images/plotter_portrait/original.jpg" alt="" class="img-medium" /></p>
<p>Much like there are many different styles of drawing, there are many different ways to convert an image to a vector format. You can trace the outlines of shapes, you can do some crosshatch shading, or you can even draw everything from a single continuous line like <a href="https://www.instagram.com/p/CBBt3vNnKd-/">this piece by artist Samer Dabra</a>. The style I opted for is based on <a href="https://tylerxhobbs.com/essays/2020/flow-fields">flow fields</a>, and creates an image with a bunch of swirly, parallel lines. I really like the smooth, flowing quality of this style, and feel that it fits the original picture well.</p>
<p>With enough artistic ability and patience, one could draw something like this manually. Lacking those attributes, I chose to use this <a href="https://github.com/serycjon/vpype-flow-imager">flow imager</a> tool, which creates flow field vectorizations of image files based on parameters that you provide. This plugin is built on a cool Python toolkit called <a href="https://github.com/abey79/vpype">vpype</a> that allows people to generate and modify vector graphics with code. If we install the flow imager tool and run it on our source image with all the default settings, we get something like this:</p>
<p class="img-caption"><img src="/images/plotter_portrait/vectorize_1.png" alt="" class="img-medium" />
<code class="language-plaintext highlighter-rouge">vpype flow_img original.jpg write vector.svg</code></p>
<p>As you can see, there are a few problems. The trees in the background look like brewing storm clouds, and my family members have no eyes or mouths (or legs in the case of my dad, who looks like he’s being swept away into the ether.) It looks cool, but it’s got a really intense, foreboding vibe to it - not exactly what I want to hang on my wall. I want the final product to show my (recognizable) family and their car on a blank background. To get the computer to vectorize in this way, we’re going to need to do some tweaking and preprocessing.</p>
<p>First let’s tackle the background removal. We need to erase all the trees and junk in the background and replace it with transparent pixels. I did all of this work in Photoshop with a simple manually drawn <a href="https://helpx.adobe.com/photoshop/how-to/layer-mask.html#:~:text=Layer%20masking%20is%20a%20reversible,to%20part%20of%20a%20layer.">layer mask</a>, but you could do this easily in any image editor (<a href="https://www.gimp.org/">GIMP</a> is a classic FOSS option.) Here’s the initial image with the background masked out:</p>
<p><img src="/images/plotter_portrait/img_2.png" alt="" class="img-medium" /></p>
<p>To get this transparency processed properly when running the flow imager, we need to add the <code class="language-plaintext highlighter-rouge">-tm</code> argument. This tells the tool not to draw lines on the transparent sections.</p>
<p class="img-caption"><img src="/images/plotter_portrait/vectorize_2.png" alt="" class="img-medium" />
<code class="language-plaintext highlighter-rouge">vpype flow_img -tm img_2.png write vector.svg</code></p>
<p>Well, the background is gone, but the result is still far from perfect. There are a few spots that I missed while masking that are very hard to see in the image, but are clearly highlighted in the vector file. I’ll go back and fix this by using the Photoshop <a href="https://graphicdesign.stackexchange.com/questions/8601/how-to-remove-low-alpha-pixels-in-photoshop">threshold tool on the alpha channel</a> to make pixels below a certain level of transparency fully disappear.</p>
<p>The removed background also makes it easier to notice that the original photo is a little cropped. My grandfather is missing the top of his head, the car is missing a headlight and its tail, and everyone is missing the tips of their shoes. We’ll need to go back and draw those in manually.</p>
<p><img src="/images/plotter_portrait/img_3.png" alt="" class="img-medium" /></p>
<p>Drawing in the toes and hair just requires a rough color match and guesstimated shape. Please ignore the fact that the drawn parts are much darker. I’m writing this recap out of order - when I actually drew these pieces in, I had already done the next step of color correction, so I was matching different colors.</p>
<p>For the missing pieces of the car, we can go on google and find pictures of the same model of car from similar angles. Then, we can stitch them in using the skew, rotate, and perspective warp tools to help sell the illusion that they belong in the image. The end product doesn’t need to look perfect in image form, as the vectorization process smooths out a lot of rough handiwork. We just need to get the basic shapes and values in place.</p>
<p class="img-caption"><img src="/images/plotter_portrait/vectorize_3.png" alt="" class="img-medium" />
<code class="language-plaintext highlighter-rouge">vpype flow_img -tm img_3.png write vector.svg</code></p>
<p>Looking good! Disregarding the accidental extra-dark regions, the only problem left to tackle is everyone’s lack of facial features. We can do this by messing with the image <a href="https://helpx.adobe.com/photoshop/using/curves-adjustment.html">curves</a> until we get a good level of contrast between eyes / mouths and the surrounding skin. We can also make use of masked <a href="https://helpx.adobe.com/photoshop/how-to/adjustment-layer.html">adjustment layers</a> to only adjust the brightness and contrast of certain parts of the image (eg. to change the brightness of the people with respect to the car.) There’s a lot of trial and error here, changing values and then vectorizing the image to see if we’ve gotten everything right. Here’s the final product:</p>
<p><img src="/images/plotter_portrait/img_final.png" alt="" class="img-medium" /></p>
<p class="img-caption"><img src="/images/plotter_portrait/vectorize_final.png" alt="" class="img-medium" />
<code class="language-plaintext highlighter-rouge">vpype flow_img -kdt -ml 1 -Ml 200 -ms 1 -Ms 20 -tm --max_size 1600 -nc 0.0005 img_final.png write vector.svg</code></p>
<p>Take note that I add a couple extra flow imager arguments for this final step, mostly to modify the spacing and size of the flow lines. However, the output using these arguments is not visually far off from what you get when using the defaults. The flow imager docs are pretty solid at explaining all the parameters that are available, and you can get a lot more wild results by enabling things like the dark field multiplier, edge detection, and multiple flow field copies.</p>
<p>One thing that I found while doing this was that lines tend to overlap a lot in the darker sections of the image. This isn’t a bad thing necessarily, but it leads to some weird looking artifacts that I was afraid would show up in the final plotted piece. To solve this, I enable the kdtree search method with the <code class="language-plaintext highlighter-rouge">-kdt</code> flag, which performs a more exact check for overlapping neighbors when generating new lines. Unfortunately, this accuracy comes at the cost of much slower image processing.</p>
<p>Another thing to consider is that a new random flow field is generated each time you re-run the flow imager. This can have a significant impact on how the final vector file looks, so the quality of any given vectorization is kind of a crapshoot. The small scale of the eyes and mouths in this picture gave me a lot of difficulty, as even the slightest change in position of the generated lines can create widely varied facial expression on the subjects, or even remove facial features completely. Check out this one generation where a wormhole was formed in my dad’s face:</p>
<p><img src="/images/plotter_portrait/vectorize_weird.png" alt="" class="img-medium" /></p>
<p>The best way I’ve found to combat this is just to re-run generation a ton of times, picking the nicest looking one at the end. Each full vector image generation takes about 20 minutes with all of my chosen options enabled, so I recommend running a few of them in parallel so you can go do other stuff while you wait. There may be some way to preview the generated flow fields at a lower resolution first and then regenerate the final image with the full quality options on, but I haven’t looked much into doing that.</p>
<p>As a final step, we can use some other <code class="language-plaintext highlighter-rouge">vpype</code> commands to clean up the vector file and get it ready for printing. By running <code class="language-plaintext highlighter-rouge">vpype read vector.svg scaleto 9in 9in filter -m 1mm linesort write --page-size 11inx15in --center final.svg</code>, we can scale the vector file to the right size, center it in the middle of an 11x15 page, and filter out any segments that are smaller than 1mm at the printed size (to clean up some more of those noisy overlapping lines I described above.) With that done, we can move on to actually plotting our vector file.</p>
<h1 id="plotting-the-art">Plotting the Art</h1>
<p>I don’t own a pen plotter, so finding a way to get the finished art piece onto paper was a bit challenging. As far as I can tell, there aren’t many companies that do on-demand pen plotting like Ponoko et al. do for laser cutting and engraving. I tried to get in contact with <a href="https://www.boldmachines.com/">Bold Machines</a> in New York, but they didn’t respond to me for a long time and then told me that they were closed until the winter, which I took as a sign to look elsewhere (though hopefully they’ve reopened!)</p>
<p>My next thought was to search around for pen plotter artists and who would be willing to plot my art. Eventually I found <a href="https://shop.paulrickards.com/">Paul Rickards</a>, who agreed to help. He was super responsive and had great answers to my questions. I ended up buying two 11x17in plots for $250, which he finished and shipped out to me within a week (along with some small pieces of his own art, which was a really nice touch.) I know from my search that some artists prefer not to plot other people’s work, so I hope I’m not sending him unwanted business here, but otherwise highly recommend reaching out to him if you’re looking to take on a similar project.</p>
<p>Because I hope to make more plotter art eventually, I also considered buying an <a href="https://shop.evilmadscientist.com/productsmenu/890">AxiDraw</a> and doing the plotting myself. In the end, though, I felt I had enough junk filling up my apartment and didn’t want another clunky machine to gather dust. I also initially wanted a larger plot than the 11x17in max dimensions the AxiDraw supports, only changing my mind after I was pretty far into conversations with Paul. If I were to make a larger format plot in the future, I’d reach out to local universities and makerspaces to see if they have some sort of CNC XY table or similar I could attach a pen to. Commercial large format plotters also exist (mostly used for architectural and engineering drawings), but I think they mainly print on lighter weight, less art-focused rolls of paper.</p>
<p>Now that it’s done, I’m really happy with how the finished piece turned out. It’s a nice way to remember my grandfather, and it makes me smile every time I see it. I gave the second copy to my dad as a Christmas gift, and hope it’ll be a happy little memento for him as well.</p>
<p><img src="/images/plotter_portrait/framed_full.jpg" alt="" /></p>Cameron SunIn this post, I document all of the steps I took to turn an old picture into a piece of plotter art. I assume basic familiarity with command line tools, but otherwise hope most reasonably computer-literate people will be able to follow along and generate some art of their own.5.15 Card Game Successfully Kickstarted!2020-07-02T00:00:00+00:002020-07-02T00:00:00+00:00http://www.csun.io/2020/07/02/5.15-card-game-kickstarter<div class="video-wrapper">
<iframe width="560" height="349" src="https://www.kickstarter.com/projects/five15game/515-a-climbing-card-game/widget/video.html" frameborder="0" allowfullscreen=""> </iframe>
</div>
<p>My cousins just successfully funded their climbing-themed card game within 6 hours of launching it on Kickstarter! I first playtested the game over a year ago, and am super impressed by how far it’s come since then. I also worked with them to take a lot of their marketing photos / shoot the game promo and rules videos!</p>
<p><a href="https://www.kickstarter.com/projects/five15game/515-a-climbing-card-game">Check them out and order the game here!</a></p>Cameron SunMy cousins just successfully funded their climbing-themed card game within 6 hours of launching it on Kickstarter! I first playtested the game over a year ago, and am super impressed by how far it's come since then. I also worked with them to take a lot of their marketing photos / shoot the game promo and rules videos!Ellipsis Split Mechanical Keyboard Buildlog2020-06-14T00:00:00+00:002020-06-14T00:00:00+00:00http://www.csun.io/2020/06/14/ellipsis-split-buildlog<p><img src="/images/ellipsis_split_buildlog/DSC07163.jpg" alt="" />
<img src="/images/ellipsis_split_buildlog/DSC07176.jpg" alt="" />
<img src="/images/ellipsis_split_buildlog/DSC07177.jpg" alt="" /></p>
<p><em>PCB and Plate files <a href="https://github.com/csun/ellipsis_split_files">here</a>. Firmware <a href="https://github.com/csun/ellipsis_split_firmware">here</a></em></p>
<p>Pictured above is a finished Ellipsis Split keyboard. The Split is a spiritual successor to the Ellipsis, <a href="/2018/07/16/custom-mechanical-keyboard.html">my previous attempt at making a custom keyboard</a>.</p>
<p>Note: I don’t actually have two capslock keys - I just ran out of keycaps that fit lol. In fact, most of the nonstandard keys are mislabeled simply because I just needed keycaps that fit.</p>
<p>The initial design phase for the Split started in early December 2019, and there were a couple breaks here and there for holidays, other projects, work, etc. Designs were sent out to be manufactured early May 2020, and painting / assembly began about two weeks after. The keyboard was finished on June 11th, 2020.</p>
<p><img src="/images/ellipsis_split_buildlog/image13.png" alt="" class="img-medium" />
<img src="/images/ellipsis_split_buildlog/image4.png" alt="" class="img-medium" />
<img src="/images/ellipsis_split_buildlog/image2.jpg" alt="" class="img-medium" /></p>
<p>These are various prototype designs from fairly early in the process. I was pretty set on an integrated wrist rest, but the last design shown here is a “short” version of the keyboard with a slightly overlapping bottom plate. The intention there would be to make the wrist rest optional for people who don’t like using them. I had some cool early designs for putting the wrist rest on rails so that it could be adjustable / removable. Doing so would’ve greatly increased the height / complexity of the keyboard though, so I scrapped them.</p>
<p>All designs until the very last had toggle switches instead of the screen that made it onto the final version. I really like the tactility of toggle switches for selecting keyboard modes / acting as a quick visual indicator of what mode you’re in, but I had a really hard time getting enough vertical clearance in the design to accommodate them. I wanted to keep the keyboard fairly slim so that the wrist rest wouldn’t have to be obnoxiously thick to be close to keycap-height. Adding toggle switches in would’ve made this really hard. Ultimately, I’m glad I switched to using a screen as a mode indicator instead, as this gives a lot more freedom to experiment with things after the fact in software.</p>
<p><img src="/images/ellipsis_split_buildlog/IMG_20200509_150749.jpg" alt="" class="img-medium" />
<img src="/images/ellipsis_split_buildlog/IMG_20200508_165650.jpg" alt="" class="img-medium" /></p>
<p>Here are the bare PCBs, which I ordered from <a href="https://jlcpcb.com/">JLCPCB</a>. I think turnaround was about 5 days between me submitting the files to the boards arriving at my doorstep… from China. That’s insane. I paid for all of the cheapest options (around $40 for 5 left PCBs and 5 right PCBs), and they still managed to process my order, manufacture the parts, and ship them 6,500 miles in less than a week.</p>
<p><img src="/images/ellipsis_split_buildlog/IMG_20200520_114246.jpg" alt="" class="img-medium" /></p>
<p>Here are the plates, which I got lasercut at <a href="https://www.ponoko.com/">Ponoko</a>. These took much longer to get here than the PCBs, and were overall not super great quality. There’s some clear stairstepping on the larger rounded corners, and a small gouge on the left top plate. Pretty cheap though, and their website is easy to use.</p>
<p><img src="/images/ellipsis_split_buildlog/IMG_20200520_155503.jpg" alt="" class="img-medium" /></p>
<p>When it came time to paint, I used a rattle can acid-etch primer with Krylon Satin Jade for the basecoat color.</p>
<p>The first time I painted the plates, I messed up in a multitude of ways. I had them laying flat on a piece of cardboard on my patio on a windy day. It was also threatening to rain (as it does in Seattle), so I rushed through the process of spray painting them. As a result, you can see some marks on the bottom left plate where the paint dried onto the corrugated cardboard and ripped off. There are also spots where the wind blew debris onto the paint. That was dumb.</p>
<p><img src="/images/ellipsis_split_buildlog/IMG_20200525_140238.jpg" alt="" class="img-medium" /></p>
<p>I also tried to clearcoat the plates only two days later. This was apparently not enough time for the base coat to cure, and the clearcoat had some terrible cracking / smudging. I ended up having to strip all of the paint and start over again. Painting apparently rewards patient people. I’m not a particularly patient person.</p>
<p><img src="/images/ellipsis_split_buildlog/IMG_20200608_195808.jpg" alt="" class="img-medium" /></p>
<p>The second time around, I searched my apartment’s recycling bin for the largest box I could find, and created a little spray booth (and waited a week after painting the basecoat before spraying the clear. This worked very well.</p>
<p><img src="/images/ellipsis_split_buildlog/IMG_20200528_124100.jpg" alt="" class="img-medium" />
<img src="/images/ellipsis_split_buildlog/IMG_20200529_183104.jpg" alt="" class="img-medium" />
<img src="/images/ellipsis_split_buildlog/IMG_20200602_151945.jpg" alt="" class="img-medium" /></p>
<p>For the wrist rests, I got router templates (the black acrylic things screwed to the wood) laser cut by Ponoko as well. I roughed them out with a jigsaw, and then used a flush cut bit to route out the rest. Afterward, I dyed them, sanded back the dye to enhance the curls in the wood, and then applied a poly finish over the course of three days (I applied some patience learned in the process of painting the plates). The end result was… ok. I wouldn’t have minded a higher gloss finish, and it bothers me a little that the left side is darker than the right. Also, there was some gouging in the right side that I think was just there - I didn’t cause it. I should’ve been more careful about the part of the board that I cut the initial pieces from.</p>
<p><img src="/images/ellipsis_split_buildlog/IMG_20200610_171041.jpg" alt="" class="img-medium" />
<img src="/images/ellipsis_split_buildlog/IMG_20200610_175901.jpg" alt="" class="img-medium" /></p>
<p>Note that the left half uses an <a href="https://keeb.io/products/elite-c-usb-c-pro-micro-replacement-arduino-compatible-atmega32u4">elite-c</a> instead of the pro micro used on the right half. This was to save a little bit of money while still getting usb-c on the half that I was planning on plugging into the computer.</p>
<p><img src="/images/ellipsis_split_buildlog/IMG_20200610_122307.jpg" alt="" class="img-medium" /></p>
<p>You can see here that the connector I used to attach the screen to the main board caused some clearance issues. I ended up cutting / filing it down until it fit. I probably should’ve just used 90 degree header pins or something instead of buying a connector like this.</p>
<p><img src="/images/ellipsis_split_buildlog/IMG_20200612_022617.jpg" alt="" class="img-medium" /></p>
<p>The screen is mounted to the bottom plate, whereas the PCB itself is mounted to the top plate. This is what necessitated the use of the connector that caused the clearance issues in the first place. There’s enough clearance for the screen to be soldered directly onto the PCB, but I had a breakout board version - hence why there’s a big cutout in the main PCB.</p>
<p><img src="/images/ellipsis_split_buildlog/IMG_20200612_024244.jpg" alt="" class="img-medium" /></p>
<p>Finished product - just before scavenging the keycaps from my old keyboard.</p>
<p><img src="/images/ellipsis_split_buildlog/DSC07180.jpg" alt="" />
<img src="/images/ellipsis_split_buildlog/DSC07192.jpg" alt="" />
<img src="/images/ellipsis_split_buildlog/DSC07186.jpg" alt="" />
<img src="/images/ellipsis_split_buildlog/DSC07200.jpg" alt="" class="img-medium" /></p>
<p>I use the screen to indicate:</p>
<ul>
<li>Mac vs Windows mode - changes my bound hotkeys (eg. fn-q is cmd-q on Mac, alt-f4 on windows).</li>
<li>If the raise / lower fn keys are depressed.</li>
<li>Whether the thumb clusters are in normal mode, game mode (moves the space bar to the left and adds a bindable f-key), or all spaces mode (so that a regular person can type without having to learn all the keys).</li>
<li>Caps / Numlock.</li>
</ul>
<p>That’s it!</p>Cameron SunA collection of images documenting the process of designing and building a custom split mechanical keyboard.Simple Personal Finance Tracking with GnuCash2020-05-17T00:00:00+00:002020-05-17T00:00:00+00:00http://www.csun.io/2020/05/17/gnucash-finance<p><em>*Please see the associated <a href="https://news.ycombinator.com/item?id=23237445">Hacker News</a> comments for some interesting discussion related to this article.*</em></p>
<p><img src="/images/gnucash/ytd_food.png" alt="" /></p>
<p>I can recall in great detail the first time I learned about GnuCash. It was a sunny spring morning in Clear Creek Canyon, CO. My little white Prius, freshly dented by the previous week’s hail, was parked precariously on the thin strip of gravel separating the road from the steep, 30ft slope to the creek below. Looking down at it from beneath the rocky cliff band we were preparing to climb, it almost seemed poised to tumble into the rushing water at any moment.</p>
<p>But I wasn’t worried about that. I was too busy complaining about my troubles with tracking my personal finances. “<a href="https://www.ledger-cli.org/">Ledger’s</a> simple plaintext interface was charming at first, but it’s getting a little tedious to input transactions like that. And I’m starting to want a more advanced GUI for plotting and generating reports,” I griped, stepping into my harness. My friends’ eyes glazed over as they nodded politely in agreement.</p>
<p>And that’s when it happened. That’s when the man belaying on the route next to us turned to me and said, “you should try GnuCash.” Though shrouded by the hood of his gray sweatshirt and his tinted wraparound sunglasses, his face was the face of someone who deeply <em>cared</em> about open source accounting software. And in that instant, my decision was made.</p>
<p>I would switch to GnuCash.</p>
<p>Today, dear reader, let me be your man in gray hoodie and wrap around sunglasses. I will help you take your first steps down the path to financial enlightenment.</p>
<h1 id="wait-what">Wait… What?</h1>
<p><a href="https://www.gnucash.org/">GnuCash</a> is free, cross-platform, open source accounting software. I would like to explain to you how to use it to keep track of your personal finances, and in turn make better decisions about your spending and saving. You can also use its data to make cool graphs, like the one above.</p>
<h1 id="why-you-should-care">Why You Should Care</h1>
<p>Using GnuCash might interest you if:</p>
<ul>
<li><strong>You’ve ever wanted to quickly answer questions about your finances, like “how much of my income am I actually saving each month?” or, “how has this pandemic affected my spending habits?”</strong> In the graph above, notice when my spending on groceries overtakes my spending on takeout. It’s roughly on the date of the first stay-at-home order here in Washington. Cool, right? Those are the sorts of trends you can unearth with this data.</li>
<li><strong>You want to own all of your financial data.</strong> I know there are some online offerings out there that promise to automagically handle the tedious parts of using GnuCash, and I would encourage you to research those as well. I choose not to use any of them because I would like to have access to my data indefinitely. I don’t want that access to hinge on a company staying in business for the rest of my life, or having the goodwill to let me download it in a future-proof file format. Because GnuCash is open source and already has good cross-platform support, I feel comfortable trusting that it (or a free replacement) will still be around in N years.</li>
<li><strong>You want to promote good financial habits.</strong> Most of the value I get from using GnuCash comes from being forced to think about every single transaction I make. Stupid purchases have a lot more weight when you can see their impact on your finances as a whole. Also, by forcing yourself to log and verify all of your transactions, you’ll be much more likely to catch fraudulent activity on your accounts early before they can cause damage.</li>
</ul>
<h1 id="why-you-should-maybe-not-care">Why You Should Maybe not Care</h1>
<p>GnuCash might not be for you if:</p>
<ul>
<li><strong>A time investment of 2-4 hours a month is too much for you.</strong> Those numbers could be higher or lower depending on the complexity of your personal financial situation, but that’s roughly the amount of time I spend.</li>
<li><strong>You wouldn’t benefit from having more granular financial data.</strong> If you’re happy with just knowing that your credit card balance is less than a certain amount each month, that’s great! You don’t need GnuCash. If, however, you want to know <em>why</em> your credit card balance is a certain amount each month, you might want to read the rest of this post.</li>
</ul>
<h1 id="the-payoff">The Payoff</h1>
<p>Finance people care about returns. So in the spirit of doing finance-y stuff, I figured I’d showcase the sorts of returns you can expect from investing your time into GnuCash. Just a heads-up, all of these graphs have had the dollar values redacted (because it’s my real data), but rest assured that these would show actual amounts if you were to generate them yourself.</p>
<p>Lets’s start with some examples from GnuCash’s built-in suite of customizable reports and graphs:</p>
<p><img src="/images/gnucash/expenses_piechart.png" alt="" />
<img src="/images/gnucash/income_expense.png" alt="" /></p>
<p>These can reveal all sorts of trends at a glance, which can then be dug into using robust search functionalities built into the program. And if the built-in search and reports don’t have the functionality you need, that’s ok! We own all of our data, and can choose to save it in a variety of friendly formats (XML, SQLite, etc.) If there’s something you want to see calculated or plotted, you can always use a package like <a href="https://pythonhosted.org/piecash/">piecash</a> to query against your data and generate custom plots. That’s what I did in order to generate the reports below, which show me in real-time how close I am to meeting my monthly and yearly budgets. I’ve also released these scripts <a href="https://github.com/csun/simple_gnucash_budget_plots">here</a> so you can use them for your own GnuCash purposes.</p>
<p><img src="/images/gnucash/month_4_food.png" alt="" />
<img src="/images/gnucash/ytd_expenses.png" alt="" /></p>
<h1 id="how-money-flows-gnucash-key-concepts">How Money Flows: GnuCash Key Concepts</h1>
<p>At the highest level, you can think of GnuCash as a place to keep track of a set of <em>accounts</em>, where each account represents a source of money. For the most part, each real-world money account you own will be tracked by its own GnuCash account. Your checking account, 401k, credit card, and Venmo would all be individual accounts in GnuCash. However, not every GnuCash account necessarily represents a single real-world money account, as we’ll address later.</p>
<p>In order to keep the values in these accounts up to date, we create <em>transactions</em>. Each transaction has an associated date, description, and amount. Every time something happens that changes the amount of money that you have, you record it in GnuCash as one of these transactions. For example, if you were to go to a bakery and buy a cake, you would create a corresponding transaction in your credit card GnuCash account - like this:</p>
<p><img src="/images/gnucash/gnucash_transaction.png" alt="" /></p>
<p>This should be a very familiar sight, as it’s nearly identical to how your bank or credit card display things on your statement. When a $13.19 transaction is added to your credit card statement, your total balance increases by $13.19. The exact same thing happens here. However, there is one major difference that you need to understand:</p>
<p><strong>In GnuCash, money is never created - only transferred.</strong></p>
<p>That may sound very abstract and zen proverbe-esque, but it’s at the heart of what makes GnuCash so powerful, and is the most important thing to understand when using GnuCash.</p>
<p>We can see this principle in action in my bakery example. Take a look at the “Transfer” column: it shows that the $13.19 we’re adding to our credit card bill is not appearing out of thin air, but is instead being <em>transferred</em> from an account named “Expenses:Food:Take Out”. If you think about it, this is exactly what happens in the real world, too. That $13.19 on your credit card statement doesn’t just appear out of thin air. Behind the scenes, the credit card company is paying the bakery $13.19 (sans fees) and then taking that $13.19 from you. The money wasn’t <em>created</em>, it was <em>transferred</em> from your credit card balance to the bakery’s bank account.</p>
<p>This simple but powerful concept is called <a href="https://en.wikipedia.org/wiki/Double-entry_bookkeeping">double-entry bookkeeping</a>, and its purpose is twofold. First and foremost, it protects you from making accounting errors. There’s no way to record a $1000 payment for rent, and then accidentally forget a zero and only deduct $100 from your bank account. By definition, every single dollar needs to come from somewhere, so GnuCash will notice $900 that are unaccounted for and will complain. A state of balance is strictly enforced. Secondly, this system automatically categorizes every transaction you make. Every time we earn or spend money, we by definition know where it came from and where it went. This makes things a whole lot easier when trying to generate reports later down the line.</p>
<p>Notice, however, that GnuCash is transferring money to an account called “Expenses:Food:Take Out”, not “Bakery Bank Account”. So what gives? Well, while it is entirely possible to make a new virtual account for every real-world money account that we interact with, it just doesn’t benefit us to do so. From a budgeting standpoint, we don’t care how much we spend at Whole Foods vs Trader Joe’s, we just care about how much we’ve spent on groceries in general - so we can throw all of those transactions in an “Expenses:Food:Groceries” account (GnuCash uses colons to denote nested accounts, so this is really the “Groceries” child account of a “Food” account in the “Expenses” grandparent account.)</p>
<p>The same goes for other things as well. We don’t care about how much money we’ve spent at Amazon, we care about <em>what we’ve spent our money on when we’ve shopped at Amazon.</em> Bought a desk? Charge it to the “Expenses:Furniture” account. A whisk? That goes in “Expenses:Kitchen Supplies”. Not specific enough for you? Create an “Expenses:Furniture:Office” account, or an “Expenses:Kitchen Supplies:Metallic” account. Too much work? Just throw those transactions into the root “Expenses” account and be done with it. <em>With GnuCash, you are completely in control of how you organize your transactions.</em> How you choose to do this will affect the level of granularity you can achieve in your reports and plots, but it’s otherwise all the same to GnuCash. All that GnuCash cares about is whether or not the transactions are balanced - when money enters one account, the same amount of money must leave from another.</p>
<h1 id="getting-started-with-gnucash">Getting Started with GnuCash</h1>
<p><img src="/images/gnucash/gnucash_main.png" alt="" /></p>
<p>When you first start up GnuCash, it will pop up a little setup wizard that forces you to create some accounts. As alluded to above, you can choose to organize your accounts however you like. With that being said, there are five top-level accounts that GnuCash will automatically create, and all other accounts you create will likely fall under these top-level accounts (as can be seen in my accounts page above). These accounts are as follows:</p>
<ul>
<li><strong>Assets</strong> are things you actually own, like checking accounts, brokerage accounts, 401ks, Venmo account, cash in your wallet, etc.</li>
<li><strong>Equity</strong> is described by the GnuCash docs as “overall net worth”, but I’ve always been iffy on how that differs from Assets for my use case. As such, I mostly choose not to use it. It may serve a purpose if you’re doing some more advanced accounting or using certain built-in reports.</li>
<li><strong>Expenses</strong> are destinations for money that you spend, like groceries, rent, taxes, etc.</li>
<li><strong>Income</strong> accounts are sources of money that you receive, like salary from an employer, gifts, etc.</li>
<li><strong>Liabilities</strong> are things you owe - money that you’re going to have to pay eventually, but haven’t yet. Personally, I only use this as the parent account for my credit cards, but you could track things like personal debts or loans here as well.</li>
</ul>
<p>It is important to create these specific accounts because GnuCash treats transactions between them differently based on their account types. For example, a positive balance in a “Liabilities:” account indicates that you own a negative amount of money whereas a positive balance in an “Assets:” account indicates a positive amount. Similarly, some of the GnuCash built-in reports rely on having the right account types set as well.</p>
<p>Once you make it through the setup wizard, take a look through some of the default accounts that GnuCash has created for you. Feel free to delete the ones you won’t use, and create new ones that you will. When you’re done with that, you’ll have a bunch of accounts - all with a balance of $0. Obviously, most of your real-world money accounts have some non-zero amount of money in them, so our final step in the setup process is to change the GnuCash ones to reflect that. As we know by now, we can’t just create this money - it needs to be transferred from somewhere. GnuCash helpfully gives us a premade “Equity:Opening Balances” account to use just for this purpose.</p>
<p><img src="/images/gnucash/empty_checking_account.png" alt="" class="img-medium" /></p>
<p>Let’s say we’ve just created an empty account named “Assets:Checking Account”, and we want to set it up so that it reflects the balance of our real-world checking account, which has $1000 in it. To do this, we’d fill out a new transaction entry with the current date, a description of “Opening Balance”, a transfer account of “Equity:Opening Balances”, and a Deposit of $1000. When we enter that transaction, our account page will update to reflect our new balance of $1000.</p>
<p><img src="/images/gnucash/full_checking_account.png" alt="" class="img-medium" /></p>
<p>Now, if you open up the “Equity:Opening Balances” account, you’ll actually be able to see and edit the transaction there as well. This is GnuCash’s golden rule in action. To get the $1000 into your bank account, it needed to leave the opening balances account. As such, both accounts will display the transaction.</p>
<h1 id="logging-your-transactions">Logging Your Transactions</h1>
<p>If you’re quick, you can finish this initial setup in less than 10 minutes. Congrats! You’re on your way to a wealth of rich financial data.</p>
<p>Now comes the hard part: remembering to actually log all of your transactions.</p>
<p>In order to maintain a state of balance, you’re going to record every single transaction you make - which sounds scary, but only takes me around 2-4 hours a month split across two sessions. If you’re more disciplined than I am, you can do your logging incrementally instead of all at once. I haven’t tried that, but I’d bet it could cut the total time invested down to around an hour or so per month.</p>
<p>Being successful at logging your data all starts with having a good workflow for <em>collecting</em> the data (receipts, invoices, etc.) In practice, I handle three separate cases here:</p>
<ul>
<li>In the ideal case, I log purchases in the GnuCash mobile app. So, as soon as I pay the bill at a bar/restaurant/store, I’ll immediately pull out my phone and log the transaction. The app support both Android and iOS, and you can import your account structure from desktop so that autocomplete works.</li>
<li>Sometimes I forget to use the mobile app, so for restaurants that support email receipts (and online purchases), I prefer those over paper receipts. The next time I check my email, I immediately move any receipts to a receipts folder to get them out of my inbox (I practice <a href="https://www.fastcompany.com/40507663/the-7-step-guide-to-achieving-inbox-zero-and-staying-there-in-2018">inbox zero</a>).</li>
<li>Paper receipts are the worst-case, but many places are still paper-only. These suck because they’re easy to lose. I recommend buying a little <a href="https://www.amazon.com/JMAF-Straight-Paper-Holder-Spike/dp/B07CR7S1FQ/ref=sr_1_4?dchild=1&keywords=receipt+spike&qid=1589768035&sr=8-4">receipt spike</a> off of Amazon for $6, and sticking any amassed paper receipts on it as soon as you get home (which has the added benefit of keeping them in chronological order).</li>
</ul>
<p>Next comes the actual data entry. The first step here is to import any transactions logged from GnuCash mobile. The app has a nice feature where it remembers the last time you exported, and then will only dump new transactions to any number of file formats, which can then be ingested by the desktop version of the app. Once this is done, it’s just a matter of hammering away at entering all the receipts you’ve saved. If you remember the steps we went through for setting our opening balances, that’s essentially all you need to do for each receipt - just changing the date, description, amount, and transfer account for each one. This might sound terrible, but it’s very easy once you get into the habit. Honestly, it can be super relaxing at times. I tend to do my logging first thing on a Sunday morning - just throw a soccer game or some mindless YouTube on the second monitor and enter cruise control for a little.</p>
<p>Now, there’s a potential (large) shortcut here. Instead of saving and entering all of your receipts manually, you could just export all of your credit card/bank account transactions from your bank website, then <a href="https://www.gnucash.org/docs/v3/C/gnucash-help/trans-import.html">import them into GnuCash</a>. There are a few reasons why I don’t personally do this, though. First of all, going through each transaction manually forces me to think about how I’m spending, which is really (IMO) the whole point of using GnuCash as an individual. Secondly, I manually log my transactions because I want more granularity than the categories that my credit card company auto-assigns to each transaction (though you could always just edit these after import). Finally, auto-importing transactions from another source is also going to auto-import any potentially fraudulent charges that might be in there. Of course, you could (should) go through each one of the imported transactions and look for suspicious ones, but I feel like I’d personally be more likely to miss something that way than if I were to manually enter everything I have receipts for and <em>then</em> inspect any discrepancies. Maybe that’s just me. Regardless, these are things to consider. For what it’s worth, I imagine it would save a ton of time to do this sort of import workflow. Especially when it comes to reconciling your accounts.</p>
<p>One important thing to consider - some transactions might involve from more than two accounts. For example, some of your paycheck goes to taxes, some to your 401k, some to your bank account, etc. Instead of creating separate transactions for each of these, we can create a single <a href="https://cvs.gnucash.org/docs/C/gnucash-guide/txns-registers-txntypes.html">split transaction</a> that groups all of them together. Don’t forget that the golden rule of GnuCash still applies here - even split transactions need to be fully balanced. We’re just balancing them across multiple accounts. So, for example, if your employer pays you $1000 and $300 goes to taxes, exactly $700 must go to your bank account, or be further divided among other accounts (so long as their values sum to exactly $700).</p>
<p>Speaking of paychecks (or rent, or other things that happen periodically), you can save time by using <a href="https://www.gnucash.org/docs/v3/C/gnucash-help/trans-sched.html">scheduled transactions</a>. This feature just automatically creates templated transactions at set dates, and is really handy for anything that happens on a schedule.</p>
<h1 id="reconciliation">Reconciliation</h1>
<p>Ok. Now that we’re done entering all of our data, it’s time to <a href="https://www.gnucash.org/docs/v3/C/gnucash-help/acct-reconcile.html">reconcile</a> our accounts - basically just making sure that there’s a 1:1 mapping of transactions in our GnuCash file to transactions in our real-world accounts. You don’t have to do this for every account, or even at all if you don’t want to. A delta of a few bucks between your real-world and GnuCash accounts probably isn’t a big deal. However, if you’re off by a little bit <em>every time</em> you log, that couple bucks difference can quickly grow into much, much more. And trying to find a mistake in your accounting over a >1 month period is a pain in the ass. Trust me. If you’re going to do it at all, it’s a whole lot easier to reconcile incrementally instead of all at once.</p>
<p>To reconcile an account, start by opening up both the reconciliation window (Actions->Reconcile) and whatever corresponding documentation you have for the real-world account. In this case, let’s pretend we’re looking at an online credit card statement.</p>
<p><img src="/images/gnucash/reconcile_1.png" alt="" class="img-small" /></p>
<p>This is what it looks like when you open the reconciliation window. In the “Ending Balance” field, you should put the most recent balance displayed on your credit card statement. Then, go through each transaction on the statement and find the corresponding transaction in the reconciliation list. Click the checkbox next to that transaction. As you check each transaction, you’ll see the “Difference” field in the bottom right of the window change value. Your goal is to get this field to zero out, indicating that there’s no difference between virtual and real-world accounts.</p>
<p><img src="/images/gnucash/reconcile_2.png" alt="" class="img-medium" /></p>
<p>If you get to the end and see a non-zero difference, you should have either a) encountered a transaction that was present in your real account but not your virtual one, or b) vice versa. In the case of a), make sure that you recognize the charge, then add a matching virtual transaction. I often find that case b) only happens when a transaction has not been processed by my bank yet. In the picture above, notice how our virtual account exceeds our real one by $25. I would bet that the $25 charge for “food” just hasn’t posted to the credit card statement yet. In that case, just uncheck it. It will show up next time you enter the reconciliation dialog, whereas the other, fully reconciled charges will not.</p>
<h1 id="dealing-with-different-types-of-assets">Dealing with Different Types of Assets</h1>
<p>If you own stocks or assets in different currencies, GnuCash has tools to help you out, but your accounting will become a little harder. I don’t personally mess with any of this functionality, so I can’t help you out unfortunately. The <a href="https://www.gnucash.org/docs.phtml">GnuCash docs</a> are pretty decent though, if a bit dense and scary looking.</p>
<p>For my stocks and non-money assets, I opt not to track them super closely in order to avoid having to update the values of each stock every time. So, I have one “Assets:Brokerage:Stocks” account that tracks the rough value of all of my holdings in USD. Whenever I log transactions, I go in and adjust that real money value of my entire portfolio with a single transaction from an “Income:Stock Value” or “Income:Interest and Dividends” account. This has served me well enough, but your mileage may vary.</p>
<h1 id="hooray-were-done">Hooray! We’re Done!</h1>
<p>And that’s it! You now have an up-to-date, categorized record of all of your financial transactions. Take this time to reflect on your spending, make an informed budget, look at nice graphs, etc. Or don’t! <em>You</em> own the data, so you can be sure it’s not going anywhere, just don’t forget to back up all of your files (I keep it under version control).</p>
<p>Hope this helped!</p>Cameron SunAn overview of how I use the open source accounting program GnuCash to manage my finances and generate cool graphs.Building a (Very) Custom Mechanical Keyboard2018-07-16T00:00:00+00:002018-07-16T00:00:00+00:00http://www.csun.io/2018/07/16/custom-mechanical-keyboard<p><img src="/images/ellipsis_main.jpg" alt="" /></p>
<p>I made a mechanical keyboard! I call it the Ellipsis. This five-pound, way-too-expensive machined aluminum monstrosity was a lot of fun to make, and I wanted to share some thing that I learned along the way.</p>
<p>For the past couple years, I’ve been using mechanical keyboards with 60% layouts (no numpad, no arrow keys, no function row). There are lots of options available in this size, and it’s seemingly one of the most popular form factors among mechanical keyboard enthusiasts. However, after spending countless hours typing on keyboards of this size, I decided that I needed something just a little bit bigger. I really like the clean, compact look of 60% boards, but found myself constantly longing for dedicated arrow keys, pgup/pgdn/home/end, a dedicated tilde key, etc.</p>
<p>Additionally I wanted to experiment with putting more keys under my thumbs. The way most keyboards are laid out, you can only comfortably access three keys with your thumbs from the home position. Seems inefficient, right? There are keyboards like the Planck, Ergodox, and Kinesis Advantage2 that allow you to get more use out of your thumbs, but I had issues with all of them (Planck is too small, Ergodox is ortholinear, Kinesis is… just too weird looking).</p>
<p>Finally, I wanted a keyboard that I could reflash with my own firmware / layouts. This was another strike against the Kinesis, and really only left me with the option of building a keyboard from scratch. I searched for a long time for a DIY kit that fit the bill, but couldn’t find anything. So, I decided to go fully custom.</p>
<h1 id="designing-the-layout">Designing the Layout</h1>
<p>The first thing I did was design the key layout for the keyboard. As this was the main thing preventing me from just buying an existing model, I wanted to make sure I got this right. If I had more time and/or patience, I probably would’ve 3D printed some prototypes to test key placement and whatnot. However, I found it much cheaper and only slightly less effective to just draw a scale version of the layout on pieces of cardboard and move things around until they felt right. Once I had rough dimensions from this process, I started drawing everything out in Adobe Illustrator.</p>
<p>There’s a great, open-source tool for creating layouts of keyboard designs at <a href="http://www.keyboard-layout-editor.com/">keyboard-layout-editor.com</a>. Unfortunately, my layout required all sorts of weird angles and precise dimensions, hence the decision to do most of the work in Illustrator. I did, however, export a mockup from the layout editor into a vector format, which I then used as a starting point for my Illustrator file. This was handy for making sure the keyswitch and stabilizer footprints were correctly sized and spaced.</p>
<p><img src="/images/ellipsis_top.jpg" alt="" /></p>
<p>Here’s a top-down picture of the keyboard so you can get a sense of the final layout. Most of the keys are in their normal places, with a few notable exceptions. In the number row, notice that the escape key has taken the place of tilde, and the backspace key has been split into tilde and delete. I have control mapped to where capslock normally is, and also have the home and end keys at the ends of that row, in easy reach of the index fingers. This is made possible by the fact that I split the keyboard into two halves, which I figured would let me position my shoulders and wrists in a bit wider and more natural way. The arrow keys and pgup/pgdn also take advantage of this fact, and can be accessed easily by both hands as a result. Interestingly enough, I seem to have placed the ‘B’ key on the wrong side of the split (I apparently used to hit it with my right hand), but adapted to that pretty quickly. Finally, the “MENU” key on the right side acts as a function key, which turns certain keys into media player controls, volume, etc.</p>
<p>For the thumb keys, my current (macOS) mapping from left to right is as follows: meta, shift, alt, enter, space, backspace. The left and middle toggle switches at the top of the keyboard let me switch between different layouts. When switched to the “up” position, the left toggle moves the enter/space/backspace cluster to the left thumb keys so that you can use the keyboard with one hand for gaming. The middle toggle is a three-position for Linux, macOS, and Windows layouts, though I realized after the fact that Linux and Windows keyboards have the same layout… The right switch is a momentary that resets the Teensy so you can flash new firmware.</p>
<p>Once the layout was done and triple-checked, I was ready to think about actually making the thing.</p>
<h1 id="change-of-plans">Change of Plans</h1>
<p>Most mechanical keyboard kits come in two or three parts. Usually, the keyswitches are mounted on a thin metal plate. The switch contacts are then usually soldered into a PCB. Some keyboards eschew the plate entirely and just mount the switches directly to a PCB. Finally, this assembly is mounted into a plastic/metal/wooden case to protect the electrical underside. There are exceptions, of course, but this is a proven, sturdy, cheap way to make a keyboard. This is initially how I planned on making my keyboard. I was going to waterjet a plate with my custom layout, then get someone to CNC mill a nice case out of wood.</p>
<p>I ran into some issues with this. Getting the plate waterjet was no problem. There are plenty of services online that’ll do it for cheap. As I researched more, though, I began to realize that making a nice wooden case could be problematic. First of all, sourcing a single slab of wood big + thick enough to machine the case out of would’ve been tricky and somewhat expensive. I guess there was the option to buy a piece of wooden countertop and machine that. Actually, I could’ve gotten a guitar body blank as well… Looking back, maybe I didn’t spend enough time thinking about this. But I digress. My other concern was with getting someone to machine the specific piece of wood that I had picked out. Most of the big online CNC milling services let you pick from a limited set of stock materials, so I wasn’t sure I’d be able to find a place where I could just send in my own stock and have someone machine it. In the end, though, this is exactly what I ended up doing with the case I made, so turns out this wasn’t actually an issue either. So I guess the lesson here is that you can definitely get a custom machined wooden case if you want one. I was just worrying too much. Worth noting that you can also contact your local makerspace to see about doing the machining yourself, but that’s a path I didn’t want to go down for various reasons (laziness, mostly).</p>
<p>As I was considering all these concerns, I started to realize that most of the custom CNC places mostly work with machining metal, and that when all was said and done, getting something machined out of metal isn’t much more expensive than getting it machined out of wood (at this quantity, most of the cost is in labor and machine setup, not materials). So I started thinking about all the cool ways I could change my design with a metal case. It struck me that I could do away with the plate entirely by just machining the holes for the keyswitches directly out of the case - much like a “unibody” MacBook Pro. I thought that idea was really cool, so I set out to design my very own “unibody” aluminum mechanical keyboard.</p>
<h1 id="designing-the-case">Designing the Case</h1>
<p>Now set on designing a very complex case, I needed to make the switch from Illustrator, a 2D graphics program, to a more serious 3D CAD tool. I had dabbled in CAD stuff before, but only on friends’ computers. So, I needed to find some software I could use on my own machine. I realized a little too late that Fusion 360 is free for hobbyists, which would’ve probably worked just fine. In the end, though, I ended up using <a href="https://www.freecadweb.org/">FreeCAD</a> for all of my design work. I’m not sure this was the best decision. It’s great that it’s free and open-source, but, as is often the case with free software, it lacks a lot of the polish of commercial offerings. Sometimes it’s just downright buggy. More on that later.</p>
<p>This might not be a popular opinion among keyboard snobs, but I actually really like my Late 2013 MacBook Pro’s keyboard ergonomics. I like having my keys flat on the same plane as my wrists. This is in stark contrast with all the other mechanical keyboards I own, which are angled slightly upwards with a >20mm vertical distance between the tops of the keys and the desk. Additionally, all of my other keyboards have sculpted keycap profiles, which means that the number key row is the tallest, and the keys are sloped in a sort of concave way that’s supposed to be ergonomic (but actually just feels really weird to me). Take a look at the DCS and OEM profiles in <a href="https://www.reddit.com/r/MechanicalKeyboards/comments/2v9zf5/keycap_profiles/">this diagram</a> - those are both sculpted profiles, and comparable to what I was using before this.</p>
<p>With this in mind, I designed the case with a couple special features. The most prominent of these features is the integrated wrist rest, the dimensions of which are based heavily on the MacBook’s. Unlike the MacBook, however, the entire bottom edge of the keyboard is filleted so that it doesn’t cut into your wrists, as I’ve found my MacBook sometimes does when typing at a certain angle. I also inset the surface where the keyswitches are mounted so that it’s a couple millimeters lower than the plane of the wrist rests. This is so that the tops of the keycaps are a little closer to being flush with the rest of the case. Because we’re working with tall mechanical keyswitches + keycaps instead of chiclet keys, it’s not feasible to get them all the way flush, but the inset definitely helps a little. Finally, I made sure to buy a set of DSA profile keycaps, which are not sculpted, and instead have a consistent height across all rows (see linked diagram above to compare them to sculpted profiles).</p>
<p><img src="/images/ellipsis_underside_nokeys.jpeg" alt="" /></p>
<p>In terms of CAD, the underside of the case was the hardest to make. The “plate” portion of the case needed to be machined down to only 1.5mm thick, so I had to design thicker sections to run throughout the middle of the case to give some support to the plate. Additionally, I needed to add cutouts for mounting the toggle switches, the usb port, threaded holes for screwing the bottom of the case to, weight reduction holes, etc. I also needed to go back a couple times to fix <a href="http://blog.inventables.com/2014/06/learn-about-milling-inside-corner.html">90 degree inside corners</a> that I’d missed. This was all made much much more annoying by a) not really knowing how to use CAD software and b) FreeCAD running incredibly slow, crashing, having weird bugs, etc. I eventually overcame all of those issues, though, and had a CAD file that I was ready to send out to machine shops for quotes.</p>
<p>One last issue I had with FreeCAD - when it came time to export to a STEP file (a non-third-party-specific CAD file format) of my final design, FreeCAD kept exporting corrupted garbage. I ended up having to export an earlier, non-corrupted version of the case, then sent it to a mechanical engineer friend to finish in Solidworks and re-export. Not great.</p>
<h1 id="getting-stuff-made">Getting Stuff Made</h1>
<p>I designed the case to be CNC milled from a single piece of aluminum. This meant minimizing the number of faces that needed to be milled (keep costs low), designing with inside corner radii in mind, etc. The biggest issue with getting the part milled was with the inside corner radii of the actual keyswitch mounting holes.</p>
<p><a href="https://en.wikipedia.org/wiki/Cherry_(keyboards)#Cherry_switches_in_consumer_keyboards">Cherry MX footprint keyswitches</a> specify a max mounting hole corner radius of 0.3mm. To meet this tolerance, you would need to cut all mounting hole corners with a <0.6mm diameter bit (read: very small & slow & costly). I believe this to be below the minimum bit size of many CNC machine shops. There are ways to get around this by cutting past the corner with a larger bit, but that could potentially affect structural integrity, definitely affects aesthetics, and would’ve required me to mess around more with FreeCAD modifying hundreds of tiny corners. I don’t know if my machinist would’ve done this part for me if I had asked (maybe they have automated tools to do it), but it surely would’ve cost me more. Additionally, certain parts of the case (the mounting holes for the switch stabilizers) were small enough that cutting past the corner in this manner would’ve completely destroyed these small features.</p>
<p>So, I came up with a plan to get around this. Basically, I ordered a case without any keyswitch holes from the CNC milling place, then sent it to a waterjet place to cut the keyswitch holes. This added a lot of time and cost to the process, but allowed me to take advantage of the small kerf of a waterjet cutter to get tighter inside corners for the keyswitch holes. In the end, though, the waterjet could only cut corners with a radius of 0.5mm, but this didn’t affect my ability to mount the keyswitches at all. I think that Cherry super over-toleranced that dimension on their spec sheet - just eyeballing my keyswitches, I was very skeptical that they required such precisely square corners. Worst case, I was prepared to go at the switches / case with a file, but in the future I think I could get away with an even larger radius for the corners.</p>
<p>After getting a bunch of quotes, I went with <a href="http://parts-badger.com/">Partsbadger</a> in Wisconsin for the milling and <a href="http://coloradowaterjet.com/">Colorado Waterjet Company</a> for the waterjet (I wanted to stay local in case I needed to pick-up / drop-off in person). Both companies did a great job with communication and finished the part earlier than the quoted lead time. The Colorado Waterjet Folks even let me + a friend tour the facility when we went to pick up the part. Partsbadger missed some small spots when beadblasting the part, but they were all minor and on the underside in non-visible places. Additionally, I got a quote that was like 30% cheaper from a company that outsources their machining to China, but it was just a little late and I had already paid for Partsbadger, so ended up staying domestic.</p>
<p>Finally, I went through <a href="https://www.pololu.com/product/749">Pololu</a> to get a clear acrylic base lasercut (so you can see the wiring through the underside of the case). This part was cheap and fast, but is a couple millimeters off the dimensions of the rest of the case on multiple sides. I’m not sure if the tolerances are just that much looser with lasercut acrylic or if Pololu did a bad job.</p>
<p>Here’s a top view of the final case with most of the keyswitches uninstalled:</p>
<p><img src="/images/ellipsis_noswitches_keyholes.jpg" alt="" /></p>
<h1 id="buying-parts">Buying Parts</h1>
<p>The biggest decision in buying parts what my choice of keyswitch. I like Cherry Browns, but wanted something heavier and a bit more tactile. I opted for <a href="https://novelkeys.xyz/products/kailh-pro-switches?variant=3747975921704">Kailh Pro Purples</a>, which supposedly feel like just that. I didn’t get to try them first, which could’ve been bad because it’s really hard to change out keyswitches on this keyboard, but luckily I like them. I used a spare Cherry Green on the escape key just to mix things up.</p>
<p>I bought my <a href="https://www.originativeco.com/products/dsa-penumbra">DSA Penumbra</a> keycaps used on <a href="https://www.reddit.com/r/mechmarket/">/r/mechmarket</a>. Buying used lessens the sting of spending hundreds of dollars on little pieces of plastic. I chose this color scheme to match the solarized color scheme of my terminal (yes, I’m a dork).</p>
<p>I picked up the toggle switches, diodes, Teensy 2.0 microcontroller, and little USB port on DigiKey. I needed to order these before finishing the case so that I knew the exact dimensions of the mounting holes.</p>
<h1 id="putting-it-all-together">Putting it all Together</h1>
<p>There are lots of <a href="https://docs.qmk.fm/#/hand_wire">good</a> <a href="https://geekhack.org/index.php?topic=87689.0">guides</a> to hand-wiring keyboards out there. It’s not hard, just really fucking tedious. <a href="https://www.amazon.com/IRWIN-VISE-GRIP-2078300-Self-Adjusting-Stripper/dp/B000OQ21CA">One of these</a> wire stripping tools was essential to finishing this process in less than a billion hours. I 3D printed a little Teensy holder to secure the microcontroller to the case, then epoxied both that and the USB port down using some epoxy putty. I stripped a USB cable and soldered it directly to the USB port, then ran that cable to the microcontroller.</p>
<p>Programming the firmware was easy, thanks to an open-source project called <a href="https://github.com/qmk/qmk_firmware">QMK</a>. My custom firmware can be found <a href="https://github.com/csun/ellipsis_keyboard/tree/master/firmware">here</a>.</p>
<p>Here are some pictures of my terrible soldering job:</p>
<p><img src="/images/ellipsis_underside_bare.jpg" alt="" />
<img src="/images/ellipsis_underside_covered.jpg" alt="" /></p>
<h1 id="the-future">The Future</h1>
<p>I had a lot of fun doing this, and hope that people enjoy the project. I’m personally all keyboarded-out for now, but I do think that there are some improvements to be made on the design if anyone is interested in continuing work / making their own Ellipsis. I’ve uploaded all of my CAD files and licensed them under a suitably permissive license in the same <a href="https://github.com/csun/ellipsis_keyboard/tree/master/cad">repo as the firmware</a>. It’d be really cool to see some Rev. 2 Ellipses in the wild :)</p>
<p>The most pressing thing left to do is to design a PCB for the keyboard. Soldering all that stuff point-to-point was a major pain, and would probably be a huge turnoff to someone who just wants to use the keyboard.</p>
<p>In terms of layout, I’m actually pretty happy with how it turned out. After a couple weeks of use, I’m still adapting to the layout but generally find it pretty comfortable now. The only thing I think I’d change is the spacing between the thumb keys, as you have rotate your thumb quite far to get between them all. I would also make the fillet on the edges of the recessed key area shallower than 90 degrees, as I think that would look better (that’s how the MacBook keyboard is designed).</p>
<p>In terms of manufacturing, I would try to figure out the actual maximum corner radius for MX switches, then cut all the keyswitch holes during the CNC milling process. Having to separately waterjet those holes is slow, costly, imprecise, and overall would be a big blocker to producing a lot of these. Additionally, I would anodize the case in addition to beadblasting it. I didn’t realize that this was possible at the time, and I’ve already scratched through the beadblast-only finish of my case in a couple spots. The case could also use some better weight reduction, though it currently has the added benefit of being usable as a weapon if need be.</p>
<p>I’m not super into lights on my keyboard, but there’s a lot of potential for really cool underlighting with the clear acrylic bottom. There’s also potential for a wood or even metal bottom to the case, which might look better.</p>
<p>That’s all for now!</p>Cameron SunI made a mechanical keyboard! I call it the Ellipsis. This five-pound, way-too-expensive machined aluminum monstrosity was a lot of fun to make, and I wanted to share some thing that I learned along the way.Generating Synthetic Computer Vision Training Data2017-08-31T00:00:00+00:002017-08-31T00:00:00+00:00http://www.csun.io/2017/08/31/synthetic-cv-dataset<p>I’ve spent the past six or so weeks working on a system that can generate realistic, labeled computer vision datasets from virtual scenes. This project was inspired by my time spent working at <a href="http://www.piaggiofastforward.com/">PFF</a> earlier in the summer, a lot of which centered around creating a simulated testing environment in <a href="http://gazebosim.org/">Gazebo</a>. Access to Gazebo reduced our testing dependencies on the limited amount of prototype robots we had around, and I feel that it was generally helpful. However, I also found that the simulation did not help very much for testing the computer vision aspect of our product, which still relied heavily on manually gathered data. I felt that simulation offered a great avenue for automated generation of this type of data, and had the added benefit of easy access to ground truth values, which were almost never available when collecting data from physical sensors.</p>
<p>So, after finishing my internship and taking a little vacation, I decided that I would explore this sort of technology on my own. I wasn’t necessarily after results - I just wanted to learn about how a system like this might fit into a computer vision training pipeline. I had never done any real work with neural networks before, so this was a good excuse to learn about those, too. With my graduation and first full-time job (hopefully in CV/robotics) looming on the horizon, I figured a little extra domain knowledge here could go a long way.</p>
<h1 id="project-goals">Project Goals</h1>
<p>As this was primarily an exploratory project, I started with some fairly broad goals. First and foremost, I wanted render realistic data for supervised training of some sort of computer vision model. Secondly, I wanted to show that my method could generate data of comparable quality to stuff produced in the “real world”. I started in the last week of July and gave myself until the beginning of the semester to get a finished product.</p>
<h1 id="selecting-a-task-and-dataset">Selecting a Task and Dataset</h1>
<p>To start this project, I needed to pick a task to train my model for, as well as a respected dataset to use for training of a baseline. Semantic image segmentation (labeling each pixel of an image based on a predefined set of classes) seemed to be a good fit here, as I had a hunch that generating synthetic data for this sort of task would be relatively straightforward. Other than the actual render, I would only need to output an identically sized, single channel image with each pixel representing a label. I figured that this could be done by an existing rendering engine without many modifications, and some searching of the <a href="http://blender.org">Blender</a> documentation confirmed this.</p>
<p>Choosing the dataset was a little trickier. Initially, I wanted to do something with outdoor scenes from a car’s perspective. I even found a <a href="http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamVid/">good dataset</a> for this sort of project. However, the task of finding, placing, and animating models of pedestrians, trees, cars, etc. in a dynamic urban environment seemed a little too difficult given my time constraints. I feared that taking on this sort of thing would quickly devolve into a computer animation project, which would’ve been cool, but wasn’t what I was going for. Still, it would be awesome to make an entire city with something like <a href="http://cgchan.com/">this</a> and generate hours of perfectly labeled driving data. The pinnacle of this sort of platform would probably look a lot like the OpenAI GTA V integration that has since <a href="https://github.com/openai/universe/issues/126">mysteriously disappeared</a>.</p>
<p>Alas, to keep things manageable, I settled on finding semantic segmentation datasets of static, interior scenes. <a href="http://groups.csail.mit.edu/vision/datasets/ADE20K/">ADE20K</a> was the first one I stumbled upon, but I found that it had too wide of a variety of scenes, and went far too granular with its labeling by separating out individual body parts and the like. I then discovered that ADE20K is partially built on the <a href="https://groups.csail.mit.edu/vision/SUN/">MIT SUN</a> dataset, which is in turn partially built on the <a href="http://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html">NYU Depth Dataset V2</a>. Compared to the other datasets, the NYU is very focused in terms of the types of depicted objects. Most of the images feature interior scenes in houses and offices with no humans present, and objects labels are simple and often fairly broad (chair, wall, table, and so on). Because I knew that I could easily find free, premade assets for these kinds of scenes online, I decided that the NYU dataset would be a perfect fit for a baseline dataset.</p>
<h1 id="training-and-assessing-the-baseline-model">Training and Assessing the Baseline Model</h1>
<p>With dataset and task in hand, I set out to train a baseline segmentation model to measure the “realness” of my synthetic dataset against. To ensure that the potential performance of my model would be consistent with modern best practices, I opted to use <a href="https://github.com/alexgkendall/caffe-segnet">this Caffe implementation</a> of <a href="http://mi.eng.cam.ac.uk/projects/segnet/">the SegNet architecture</a>. It had decent documentation, and someone had even trained it on the SUN dataset before, so I had no doubt that it could perform decently on the NYU dataset.</p>
<p>Training the model was easily the hardest part of the project. My lack of experience with neural nets meant that I had to lean heavily on trial and error to get things working. Even when things started working, I often had no intuition if my results were “good” or not. What is a reasonable learning rate? What sort of accuracy should I be looking for after 10 epochs? After 100 epochs? How long should I let the thing train for before shutting it down? These sorts of little questions were often major roadblocks, as every little tweak would require restarting hours of training in order to even see any effects. It was maddening to know that most of these questions could probably have been answered by a domain expert in less than a minute, saving me literal days of effort. I usually enjoy learning this sort of stuff on my own, at my own pace, but this whole process really had me wishing for easy access to a mentor or adviser.</p>
<p>Speaking of wasted time, man did I ever underestimate how resource-intensive this stuff is. I went into this project thinking that it would be a good way to get some use out of my newly assembled computer’s GPU. I spent a day trying to get the provided GPU-enabled docker image working on my machine, only to find that I don’t have enough VRAM to even run training at the smallest batch size with downsampled images. After this, I ran the horrendously slow CPU-based training for about a day before deciding it wasn’t worth it. With my system maxed out, I would’ve needed to train for something on the order of weeks in order to process the number of epochs suggested by the tutorial. Finally, I decided to use my $300 of account credit on Google Cloud Platform to rent an instance with a GPU. I never thought I would be pleased to see a program taking 48 hours to run, but in this case, it was a huge relief.</p>
<p>Initially, I split the dataset of ~1400 images 80/20, using the first 80% of images for training and the rest for testing. I also wrote some scripts to extract all images and labels into formats that SegNet liked, removing all but the 30 most prevalent labels (down from around 900). At this point, the least common label was only present in around 12% of the training set, which I thought was a decent cutoff.</p>
<p>Then came the tweaking. It was always little things, like the fact that the NYU dataset uses 0 as its “unlabeled” label whereas SegNet requires the unlabeled pixel value to be greater than all others. Or realizing that objects that are neither unlabeled nor one of the 30 most common should be considered unlabeled, not simply lumped into a giant “other” label (in hindsight, this was a dumb mistake). None of these changes took more than a couple minutes to actually implement, but the fact that they were followed by batch processing of 1400 images + hours of training was really a killer. Other small changes included reducing the number of classes to 16 (to improve average accuracy), and shuffling the data before dividing into testing and training sets (as images of the same type of location were often clumped together).</p>
<p>Finally, I managed to get a result that I was happy with. I stopped training at 240 epochs (40,000 iterations with batch size 7), though it looked like the model could’ve continued to improve with some more training. This resulted in an average class accuracy of 94.51% and a global accuracy of 56.11%. I was a little confused by the fact that average class accuracy is <em>higher</em> than global accuracy - something that didn’t happen in any of the results published in the SegNet paper. This led me to believe that I had written the accuracy grading scripts wrong, but after triple checking, I’m pretty sure that I’ve written them correctly. My theory is that, because accuracy weights true negatives and true positives equally, less common labels will tend to appear highly accurate even if there are never any predicted positives. For example, the least common label, “sink”, had 99.56% class accuracy, but it could be the case that the model simply learned to never predict sink, causing the massive amount of true negatives to mask the minimal amount of false ones. A better breakdown of precision/recall would probably be good here.</p>
<h1 id="generating-synthetic-data">Generating Synthetic Data</h1>
<p>After getting intimately familiar with the model input format and the types of images in the dataset, I started work on generating the synthetic images. Because I had a little previous experience with it, I planned on doing all of the rendering with Blender and the Cycles rendering engine. Initially, my plan was to find free models of furniture online, then arrange them into scenes. However, after a little poking around, I realized that I could get entire interior scenes for free. This saved me from spending a lot of time fixing compatibility issues with imported models, lighting scenes, and doing other little things. I selected about six scenes of various rooms that one might find in a normal-looking house.</p>
<p>Because the scenes were pre-lit and everything, rendering was easy. For each scene, I would animate the camera to jump between three or four angles, squeezing as many varied images as possible out of each scene. To output the label images I needed, I made use of Blender’s built-in <a href="https://docs.blender.org/manual/ja/dev/render/blender_render/settings/passes.html">render passes</a> - specifically, the object index pass. This pass lets you assign each object a “pass index”, then outputs an image where each object in the scene is masked by its pass index value. I wrote a small Blender add-on that let me assign selected objects the correct pass index for their label, then went through all scenes assigning the labels to the relevant objects. Sometimes, I would need to separate some pieces of meshes out into separate objects (like separating closet doors from the rest of the closet mesh) so that I could accurately label the different parts. Once this was done, I would run another Blender script to set the correct image dimensions, output locations, and do other little housekeeping things. Because Blender’s scale for pixel values is between 0 and 1.0, but the object index pass outputs from 0 to NUM_LABELS, the script also adds a compositing node to divide all the object index values by 255. This way, once the images are saved, each label pixel has the proper value between 0 and NUM_LABELS. Once a scene has been processed like this, all that’s left to do is render the animation. The images below are an example of some renders and their (colorized) labels. You can view the rest of the raw, generated images and labels <a href="https://github.com/csun/syntrain/tree/master/model/syn_data/images">here</a>.</p>
<p><img src="/images/syntrain_labels_sample.png" alt="" /></p>
<h1 id="measuring-synthetic-data-quality">Measuring Synthetic Data Quality</h1>
<p>Due to time constraints, I only ended up rendering about 20 synthetic examples. This represents about 10% of the original testing dataset, and a measly 1.x% of the training set, so I realize that results here probably do not say much - especially the retraining results.</p>
<p>When using the baseline trained model to predict labels for the synthetic examples, it achieved an average class accuracy of 90.1% and a global accuracy of 28% - both much worse than the performance on the original training set. For real world applications where this data is only used for training, these results don’t really matter. However, they are certainly concerning, as any difference between the synthetic and real images that is drastic enough to cause this much of a performance hit might also hurt training effectiveness. Honestly, I’m not really sure what could’ve caused this disparity. Maybe the differences in chosen camera angles and parameters? I’ve noticed that these renders have a much more cinematic look to them than the images in the real dataset. The scenes are much more sterile and well-lit. I tried to select scenes where this was much less prevalent, but maybe I could’ve done a better job.</p>
<p>I also trained another SegNet instance on a combination of my images and the original training set. After running training for the same amount of iterations that the baseline model went through, I used this new model to predict the original testing dataset. This model performed better than the baseline in both average class accuracy and global accuracy, but by a very small amount (~0.02% and 0.2% respectively). This is certainly not a statistically significant result, but I wasn’t expecting it to be. I simply didn’t add enough synthetic images to the training set to make any meaningful difference.</p>
<p>I do have reasons to believe that I was on the right track, though. During the early stages of the project, I actually found a paper by a group of researchers who recently worked on <a href="https://joonyoung-cv.github.io/assets/paper/17_cvpr_physically_based.pdf">something very similar</a>. They claim to beat state-of-the-art semantic segmentation results by pre-training on synthetic images, then fine-tuning with the NYU dataset. These results are quite promising, and I think that their synthetic images are comparable in quality to mine. In fact, I think that many of my images are more realistic-looking than theirs, which gives me hope that I would see similarly successful results were I to create a larger dataset.</p>
<h1 id="conclusion">Conclusion</h1>
<p>So, despite the fact that I didn’t get any significant results or do anything groundbreaking, I would call this project a success. I learned a good amount and more or less achieved my project goals. Yay!</p>
<p>In terms of real-world applications of this tech, I’m not sure that this specific use case of general-purpose semantic segmentation makes much sense. With my Blender-based workflow, it only takes about 5 minutes to label an entire scene, which can then be used to generate quite a few images. However, this does not account for the time it took the original author to make the scenes, or the time it takes to render. If you want to use these techniques in an environment where you’re also creating the scenes yourself, it’s worth noting that it might not actually be any faster than manually labeling real pictures.</p>
<p>In my opinion, these techniques really start to become attractive when you need data that humans have difficulty manually generating or labeling. When paired with real-time simulation, image generation like this could let mobile robots spend thousands of hours controlling their own actions in a realistic environment, learning to identify and avoid obstacles without any risk of harm. Self driving cars can learn about situations that are too rare or costly to get good training data on, like avoiding freak accidents or unpredictable drivers. One thing that I’d be really excited to apply this to is the creation of a labeled point cloud dataset, as understanding lidar data is becoming more and more important. For both point clouds and images, one could generate all sorts of data that can’t realistically be obtained in the real world, like ground truth for normals, velocities, readings from reflective or translucent materials, and so on.</p>
<p>All things considered, I think that this sort of technology has massive potential, and I’d be very excited to work on something like this full-time. If you or someone you know is working on this kind of simulation for robotics or computer vision, I’d love to chat (especially if you’re hiring!)</p>
<p>If you’re interested in looking at the code written for this project, you can find it all <a href="https://github.com/csun/syntrain">here</a>.</p>Cameron SunI’ve spent the past six or so weeks working on a system that can generate realistic, labeled computer vision datasets from virtual scenes. This project was inspired by my time spent working at PFF earlier in the summer, a lot of which centered around creating a simulated testing environment in Gazebo. Access to Gazebo reduced our testing dependencies on the limited amount of prototype robots we had around, and I feel that it was generally helpful. However, I also found that the simulation did not help very much for testing the computer vision aspect of our product, which still relied heavily on manually gathered data. I felt that simulation offered a great avenue for automated generation of this type of data, and had the added benefit of easy access to ground truth values, which were almost never available when collecting data from physical sensors.