Nicolas314https://maz.one/2024-02-15T14:00:00+01:00C'est là ma zoneTime Machine recipe2024-02-15T14:00:00+01:002024-02-15T14:00:00+01:00nicolas314tag:maz.one,2024-02-15:/time-machine-recipe.html<p>Objective: backup a Mac to a samba share.</p>
<p>For reference: I am running MacOS Sonoma 14.3.1 and default Samba (2:4.17.12) on Debian stable (Bookworm).</p>
<p>On the Samba side, create a new share and add a section to <tt class="docutils literal">/etc/samba/smb.conf</tt></p>
<pre class="literal-block">
[Time Machine]
path = /data …</pre><p>Objective: backup a Mac to a samba share.</p>
<p>For reference: I am running MacOS Sonoma 14.3.1 and default Samba (2:4.17.12) on Debian stable (Bookworm).</p>
<p>On the Samba side, create a new share and add a section to <tt class="docutils literal">/etc/samba/smb.conf</tt></p>
<pre class="literal-block">
[Time Machine]
path = /data/tm
fruit:time machine = yes
writeable = yes
public = yes
guest ok = yes
force user = joe
</pre>
<p>Set up the correct <tt class="docutils literal">path</tt> and <tt class="docutils literal">force user</tt> to reflect the adequate local directory and user who owns that directory, and restart samba.</p>
<p>On MacOS, open Time Machine settings and add the new share. You need to be on the same local network. You can set up a password for that backup (recommended). Start the backup immediately and check that it works.</p>
<p>Tip: to speed things up on Mac OS you can increase Time Machine priority by issuing this command in a terminal:</p>
<blockquote>
sudo sysctl debug.lowpri_throttle_enabled=0</blockquote>
<p>and restore it afterwards with:</p>
<blockquote>
sudo sysctl debug.lowpri_throttle_enabled=1</blockquote>
<p>Hope it can be useful for somebody else!</p>
Home networking: fun with addresses2023-10-04T21:30:00+02:002023-10-04T21:30:00+02:00nicolas314tag:maz.one,2023-10-04:/ipv6-lan.html<hr class="docutils" />
<div class="section" id="the-setup">
<h2>The setup</h2>
<p>My home network is unusual, I'll grant you that. Hundreds of connected machines, servers, tablets, gadgets made me switch at some point to a <cite>/23</cite> to push the limits to 500 entries on my LAN. In my defense, I have a lot of multi-port routers, managed switches, and …</p></div><hr class="docutils" />
<div class="section" id="the-setup">
<h2>The setup</h2>
<p>My home network is unusual, I'll grant you that. Hundreds of connected machines, servers, tablets, gadgets made me switch at some point to a <cite>/23</cite> to push the limits to 500 entries on my LAN. In my defense, I have a lot of multi-port routers, managed switches, and machines that have both wired and wireless connections. Those things tend to add up quickly.
Nowadays I managed to reduce it back to 200 entries and go back to a <cite>/24</cite> but it is just a question of time until I run out of IPv4 addresses again.</p>
<p>There is one property I have been very careful with: assigning fixed IPv4 addresses to all hosts on my LAN. I just like to know that my printer will always be reachable as <cite>printer</cite>, my NAS as <cite>NAS</cite>, and my router as <cite>router</cite>. Memorizing addresses for the main hosts saved me several times when the DNS gets lazy, but also mostly when I just finished installing another OS and it didn't pick the DNS servers correctly. Call me maniac but I like to know where things are, and I want to know when there are unknown hosts on my LAN (thanks <cite>arpwatch</cite>).</p>
<p>Doing that on an IPv4 LAN is really easy: I configured a single DHCP server on the LAN with fixed leases for all known machines, done. I went one step further and declared LAN hosts to my local DNS (AdGuardHome). For my own administration I maintain a single file containing all information: MAC address, LAN address, and host name, and a short home-made script that manages to copy all relevant information where it should be. Since I tend to try out a lot of various DHCP and DNS servers, that script has grown to support multiple file formats. But after all it's just all variations around <cite>printf</cite>, nothing to be happy about.</p>
<p>You might have read in previous blog posts about my sorry adventures in the world of IPv6, trying to wrap my head around how to make it work on my home network. I got several steps further down the line of enabling IPv6 and got to the conclusion that it was never designed for networks such as my own. Let me elaborate.</p>
<p>Objective: have my printer available as <cite>printer</cite> on my LAN, on IPv6.</p>
</div>
<div class="section" id="fixed-addresses-with-dhcpv6">
<h2>Fixed addresses with DHCPv6</h2>
<p>I tried to replicate what I did for IPv4 and tried to use a DHCPv6 server. Assigning fixed addresses proved to be a lot harder than maintaining a list of MAC/IP/name triplets. IPv6 does not rely on Ethernet MAC addresses to identify DHCPv6 clients but on something called a DUID (Device Unique ID), which is a random bit string pulled out of a horror movie, maybe sometimes during OS install, or maybe when the NIC first connected to a network. Who knows? Those things seem to have a life of their own. On Linux I ended up browsing (a lot of) source code to understand who generated a DUID for that particular machine, and how to make sure it won't change when I next reboot or change something else. Turned out that on Debian it mostly depends on which piece of software handles your DHCP client. Since there are multiple <em>standard</em> pieces of software to do that, I had to become an expert in Debian DHCP clients over the ages.</p>
<p>Anyway, DHCPv6 is hated by a not-so-small part of the IPv6 community and there is apparently no way to have a rational conversation about it. Android does not support DHCPv6 and only wants SLAAC self-assigned addresses. This won't change until the guy in charge at Google gets fired or retires so don't hold your breath.</p>
<p>So I activated dual DHCPv6/SLAAC address assignment on my router. Now all Android devices work but everything else is messed up. Next task: figure out how to tell all the various operating systems on my home network to not use SLAAC if they have DHCP. Debian's multiple <em>standard</em> network management tools all have different ideas about how to do that, so back to discovering Debian DHCP clients over the ages.</p>
</div>
<div class="section" id="self-assigned-addresses-with-slaac">
<h2>Self-assigned addresses with SLAAC</h2>
<p>Ok you know what? Forget about memorizable addresses. Let's go full SLAAC. As long as all my machines keep having the same name, I am happy. It requires the DNS to be always up though, so might be a good time to think about adding a second one for redundancy.</p>
<p>SLAAC is an address assignment method where the DHCP server (a <cite>router</cite> in IPv6 speak) only provides a common prefix to the whole LAN. Every machine takes that 64-bit prefix and adds its own 64-bit made up bits to generate a hopefully unique address. Et voilà: you have unique 128-bit IPv6 addresses for everybody! Now how would I link those self-generated addresses to DNS entries?</p>
<p>Easy: you go to each machine on your LAN and patiently collect their monstrous IPv6 self-generated addresses, then write them down somewhere for your DNS to digest. Could be done with a dozen machines or so, but for 200 hosts that's a whole week-end down the drain. Life is too short.</p>
<p>Another solution would be to use mDNS, a magic protocol that allows machines to announce themselves with their name and the services they offer. This is well supported on Mac OS (which calls it <em>Bonjour</em>) but that's about it. On Linux you need to install and run <cite>avahi-daemon</cite> which has a friendly XML configuration file well-hidden somewhere in <cite>/etc</cite> and strictly no human-readable error messages when you mess it up. I configured those service files once and hope I will never have to do it again.</p>
<p>Best part: once I was done configuring mDNS on Debian machines, I found out Tasmota does not support mDNS (because reasons) and there is no chance it ever will. Good grief! That pretty much kills any attempt at having my connected plugs to work by name on IPv6. To be honest, Tasmota does not work too well with IPv6 anyway, and I cannot blame them. Time to think about something else.</p>
</div>
<div class="section" id="manual-ipv6-addresses-where-i-care-ipv4-everywhere">
<h2>Manual IPv6 addresses where I care, IPv4 everywhere</h2>
<p>The solution I settled on was to switch everything back to an IPv4 home network, with fixed addresses and names as I have always done. If it ain't broken, don't fix it.</p>
<p>I still want to enjoy IPv6 on phone, tablets, and servers I want to be reachable from outside. The solution I found is rather simple:</p>
<ul class="simple">
<li>Set up the local <cite>router</cite> to be pure SLAAC, so pure client machines work, including Android.</li>
<li>Disable IPv6 on all machines that don't need it. One less thing to worry about.</li>
<li>Use my newly-acquired knowledge about Debian network configuration to manually assign static IPv6 addresses to servers I want to make reachable from the outside world. That means hardcoding addresses in <cite>/etc/network/interfaces</cite> and maintain them.</li>
</ul>
<p>That is somewhat dirty, I know, but it works. For now. Until my ISP decides to change my delegated IPv6 prefix, or I change ISPs. This happens more often than I would like: whenever my landline dies, I connect my phone and its very generous monthly data allowance so my whole LAN is connected again with an IPv6 prefix that changes every time I reconnect the phone. For now I just gave up on having my home servers being reachable from the outside world during those short periods.</p>
</div>
<div class="section" id="conclusion">
<h2>Conclusion</h2>
<p>Running IPv6 on my home network involved a lot more work than I ever anticipated. If all hosts on your LAN are stupid smart phones and tablets, let your ISP-provided router disaster run things for you and forget about it. If you really want to get into IPv6 because it is cool and you think you may need it for your job, I would recommend finding another job.</p>
<p>There is no place like 127.0.0.1.</p>
</div>
AI-generated images2022-09-01T23:00:00+02:002022-09-01T23:00:00+02:00nicolas314tag:maz.one,2022-09-01:/ai-generated-images.html<img alt="AI-generated colorful picture" src="/images/ai-generated.jpg" />
<p>If you haven’t followed, there has been a recent (Summer 2022) wave of AI tools for image generation from a text prompt. It works this way: describe what you want to see, press a button, see what you get.</p>
<p>Here are some prompts and the images that were generated …</p><img alt="AI-generated colorful picture" src="/images/ai-generated.jpg" />
<p>If you haven’t followed, there has been a recent (Summer 2022) wave of AI tools for image generation from a text prompt. It works this way: describe what you want to see, press a button, see what you get.</p>
<p>Here are some prompts and the images that were generated from them. Taken from the Dall-E2 examples:</p>
<div class="figure">
<img alt="An armchair in the shape of an avocado" src="/images/dalle2-armchair-avocado.jpg" />
<p class="caption">An armchair in the shape of an avocado.</p>
</div>
<div class="figure">
<img alt="Oil painting of a basketball player" src="/images/dalle2-basket.jpg" />
<p class="caption">An expressive oil painting of a basketball player dunking, depicted as an explosion of a nebula.</p>
</div>
<div class="figure">
<img alt="A photo of a monster" src="/images/dalle2-monster.jpg" />
<p class="caption">A photo of a white fur monster standing in a purple room.</p>
</div>
<p>How does it work? Feed an AI with billions of images and let it compress it all down to a much smaller data set of coefficients that encode all the inputs. When I say “feed” it takes a bit more than that: if you give it Van Gogh paintings you would also tag each image as ‘Van Gogh’ so it can attach a name to a style, but also a rather complete description of what is in each image, and where. That’s the hard part and it has already been done by several companies out there.</p>
<p>A few years back those tools gave us <a class="reference external" href="https://prisma-ai.com/prisma">Prisma</a> a mobile app that transforms your photos into paintings copying an artist’s style. That was very neat and fun. Here are some examples:</p>
<img alt="Original image" src="/images/prisma-before.jpg" />
<img alt="After transformation" src="/images/prisma-after.jpg" />
<p>Go one step further: instead of essentially applying a filter to a photograph, tell the AI what you want to see and let it randomly pick from its own knowledge base to generate something completely new. The results you will get are entirely dependent on the kind of images that were used to train the AI. From what I observed, Dall-E has no trouble generating photo-realistic pictures that are hard to tell apart from stock photos. Midjourney must have been trained with a lot of paintings and gothic art, because most of what I got were museum-quality paintings.</p>
<img alt="Library of Babel by midjourney" src="/images/midj-library-babel.jpg" />
<p>And of course there’s the matter of the prompt you used to generate the image. Finding the right way to express what you want can be very hard. The AI engine does not handle ambiguity very well and sometimes completely misunderstands the request.</p>
<img alt="Monster under the bed by Dall-E2" src="/images/dalle2-monster-bed.jpg" />
<p>The prompt was:</p>
<p><em>Cartoon drawing of a cute monster hidden under a child's bed, with the child awake sitting on the bed</em></p>
<p>If you haven’t tried yet, I would encourage you to go and have fun with those tools. It doesn’t require any software knowledge, just be creative with what you want to see and let it do all the legwork! So what is available today?</p>
<p><a class="reference external" href="https://openai.com">OpenAI</a> has a beta program now accessible to all. You get a small number of free credits to start with, you pay to get more, which is fair.</p>
<p><a class="reference external" href="https://www.midjourney.com/home">Midjourney</a> receives prompts from a discord server. Invites are free, same story as Dall-E: get free credits to start, buy a subscription to play more.</p>
<p>Another option is to download and run the whole software and data and do it yourself on your PC. <a class="reference external" href="https://github.com/CompVis/stable-diffusion">Stable diffusion</a> does just that and it works fine provided you have a PC running Linux with a very beefy GPU on an Nvidia graphics card with 6GB or more RAM. That’s expensive hardware and spicy electricity bills so all in all a subscription with a cloud-based AI might well be a lot cheaper.</p>
<p>Here are some images I generated using stable diffusion with the prompt:</p>
<p><em>A digital illustration of Steampunk Paris in 1900</em></p>
<img alt="Steampunk Paris in 1900 (1)" src="/images/sd-paris-1.jpg" />
<img alt="Steampunk Paris in 1900 (2)" src="/images/sd-paris-2.jpg" />
<img alt="Steampunk Paris in 1900 (3)" src="/images/sd-paris-3.jpg" />
<img alt="Steampunk Paris in 1900 (4)" src="/images/sd-paris-4.jpg" />
<img alt="Steampunk Paris in 1900 (5)" src="/images/sd-paris-5.jpg" />
<p>First question: who created those images? Is that me, human, who came up with the idea in the first place? Or is it the SD engineers who took the pain of processing billions of tagged images and I merely used their results? Or is it all the common knowledge contained into billions of images fed to a fancy data compression algorithm?</p>
<p>Let’s take an example: here’s what came out when I asked for <em>some cute monsters</em>.</p>
<img alt="Cute monsters by stable-diffusion (1)" src="/images/sd-monsters-1.jpg" />
<img alt="Cute monsters by stable-diffusion (2)" src="/images/sd-monsters-2.jpg" />
<p>Good results, right? If I had contracted out a human illustrator to do the same, I am pretty sure we would agree that all the creativity came from whoever did the drawing and the customer merely expressed an intent.</p>
<p>Getting quality pictures requires fairly elaborate prompts that are rooted in graphical designer know-how. If you want to get an idea, check out some of the <a class="reference external" href="https://dallery.gallery/dall-e-ai-guide-faq/">sites dedicated to explaining what you can do with elaborate prompts</a></p>
<p>Hard to tell who the real artist is, right?</p>
<p>A friend of mine pointed out that the same questions came up when photography started. I think everybody agrees that photography is an art created by humans despite the fact that the artist didn’t create the reality they are capturing. Yes, reality exists independently from photographers but seeing it through their opto-mechanical eyes changes everything. Think about portraits by <a class="reference external" href="https://en.wikipedia.org/wiki/Annie_Leibovitz">Annie Leibovitz</a> or <a class="reference external" href="https://www.haroldfeinstein.com/">Harold Feinstein</a> if that wasn’t obvious.</p>
<p>I don’t have the answer to that question. Right now I get the feeling that there is a consensus that generated pictures are the work of whoever came up with the prompt, but time will tell.</p>
<p>If you want to find inspiration for prompts or just have some fun, check out the SubReddits about AI-generated images. The <a class="reference external" href="https://reddit.com/r/dalle2">Dall-E2 subreddit</a> is just fantastic.</p>
<p>Here is a recent one: Mona Lisa attending a techno party, having the time of her life.</p>
<img alt="Mona Lisa having the time of her life in a techno festival" src="/images/dalle2-monalisa.jpg" />
<p>That one had me in tears! At first the face looks familiar but you just don’t know where from, and then the prompt hits: of course that’s Mona Lisa! But it’s not a painting it’s a photo, so probably something recent, she’s probably old by now (about 500 years old). Look at her again: she’s way past her youth but she still has that haircut and tattoo, and dress, with jewellery on top she must have picked for the festival. Participating in techno festivals is way past her age but she’s having the time of her life! Her smile is unequivocally sincere this time, and she’s tilting her head in an attempt to be charming, just like when she was 15 and seduced Leonardo to the point he decided to make her immortal. A picture is worth a thousand words.</p>
<p>Of course I am making up that complete backstory for that picture, but that’s the whole point behind art, right? Makes us think, imagine, feel, invent and be creative ourselves.</p>
<p>Here is another touching one: two girls pranking a boy on the phone in 1968. The tone is right, the picture is messed up as expected, the characters are perfectly composed.</p>
<img alt="Girls pranking a boy on the phone in 1968, by Dall-E2" src="/images/dalle2-girls-pranking.jpg" />
<p>I will finish with that one:</p>
<img alt="Daughter looking at her dad who just made a dad joke, by Dall-E2" src="/images/dalle2-daughter-dad-joke.jpg" />
<p><em>Girl looking at her dad who just made a dad joke</em></p>
<p>There are some fantastic guides around how to generate the best images with various services out there. I just discovered <a class="reference external" href="https://www.slaphappylarry.com/making-ai-art-with-midjourney/">Making AI Art with Midjourney</a>.</p>
<p>The look in her eyes is priceless. I can see a teenager feeling desperate about her dad’s poor jokes. There is both tenderness and regret in those eyes. Can’t imagine how a photographer would manage to capture that except by sheer luck.</p>
<p>If anyone had asked me just a few months back which jobs would never be replaced by an AI, art would have been my first choice. There is still a very long way to go until we replace graphical designers, photographers, illustrators, and design artists in general, but there is no question that those tools will change the game in very radical ways.</p>
<p>Right now I am having fun re-creating complete worlds in the style of Miyazaki so stay tuned :-)</p>
OpenWRT on Raspberry Pi 42021-11-03T21:00:00+01:002021-11-03T21:00:00+01:00nicolas314tag:maz.one,2021-11-03:/openwrt-raspberry-pi-4.html<img alt="Raspberry Pi as router" src="/images/openwrt-prompt.png" />
<div class="section" id="gbit-routing">
<h2>Gbit Routing</h2>
<p>Driving a home Gbit Internet connection requires heavy hardware. Or maybe not?</p>
<p>Following <a class="reference external" href="https://forum.openwrt.org/t/so-you-have-500mbps-1gbps-fiber-and-need-a-router-read-this-first/90305">recommendations on OpenWRT forums</a> I tried it on a Raspberry Pi 4 with an additional USB3/Gbit adapter and the results are beyond anything I could have hoped for. This is not a cheap solution …</p></div><img alt="Raspberry Pi as router" src="/images/openwrt-prompt.png" />
<div class="section" id="gbit-routing">
<h2>Gbit Routing</h2>
<p>Driving a home Gbit Internet connection requires heavy hardware. Or maybe not?</p>
<p>Following <a class="reference external" href="https://forum.openwrt.org/t/so-you-have-500mbps-1gbps-fiber-and-need-a-router-read-this-first/90305">recommendations on OpenWRT forums</a> I tried it on a Raspberry Pi 4 with an additional USB3/Gbit adapter and the results are beyond anything I could have hoped for. This is not a cheap solution but it is probably the easiest to maintain and the most versatile I have found so far, with tons of potential for cool stuff.</p>
<p>In terms of performances I can reach a solid, reproductible 940Mbps on both interfaces when talking over a LAN one-way. I haven't tried full LAN/WAN simultaneous Gbit traffic but managed to max out my Internet connection with 1000 Mbps download and 500 Mbps upload simultaneously. I am not claiming those are accurate or rigourous measurements though. I am using iperf3 without any specific options to measure speed one-way:</p>
<p>On the server side:</p>
<pre class="literal-block">
iperf3 -s
</pre>
<p>On the client side:</p>
<pre class="literal-block">
iperf3 -i 1 -c SERVER_ADDRESS
</pre>
</div>
<div class="section" id="quick-introduction-to-openwrt">
<h2>Quick introduction to OpenWRT</h2>
<p>OpenWRT is a Linux distro specialized for heavily-constrained routers. The minimum requirements are ridiculously small: eveything is optimized down in size to fit the smallest devices you can imagine. OpenWRT does an excellent job at replacing stock firmware on commercial routers from all vendors, often providing better performance and more features than the original firmware.</p>
<p>OpenWRT specializes in home routing: firewalling your home network from the Internet, adding Wi-Fi access points, connecting private networks, or driving VPN connections. I even used it to transform a connected hard drive into a Samba server. OpenWRT can be entirely configured through command-line and text config files in a familiar Linux environment. It also comes with a web-based GUI called Luci that is rather simple, user-friendly, and well-documented. What's not to love?</p>
<p>The most impressive thing about OpenWRT is the sheer number of supported devices from all architectures and sizes, and the fact that they keep supporting hardware for such a long time after those products have been end-of-lifed by their vendors. I cannot thank the team enough for all their dedication and hard work. Wonderful job, guys!</p>
</div>
<div class="section" id="shopping-list">
<h2>Shopping List</h2>
<p>To transform an RPi4 into a Gbit router you will need:</p>
<table border="1" class="docutils">
<caption>Hardware</caption>
<colgroup>
<col width="50%" />
<col width="50%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">Part</th>
<th class="head">Price</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>RPi4</td>
<td>50-100€</td>
</tr>
<tr><td>Power Supply</td>
<td>10€</td>
</tr>
<tr><td>Case + Fan</td>
<td>10-20€</td>
</tr>
<tr><td>USB3/Gbit adapter</td>
<td>10€</td>
</tr>
<tr><td>USB3 Thumb drive or SD card</td>
<td>5-10€</td>
</tr>
<tr><td>Total</td>
<td>85-150€</td>
</tr>
</tbody>
</table>
<p>Prices will vary a lot depending on what/when/where you buy. You should try to find the smallest possible RPi4, a 1GB RAM model will work perfectly fine if you can still find one (on eBay?). Having 2 or 4GB is perfectly overkill. In my case I just used a 4GB model that was gathering dust in a drawer, so I'm wasting about 4GB of RAM.</p>
<p>Power supply: the official one is graded for 3 Amperes. This is what I am using. More is better, less than that will probably not be sufficient to power the Pi.</p>
<p>Case: you <em>don't</em> want to leave your RPi4 without protection and <em>you do want to cool it down</em>. Get a case with a silent fan or one that is passively cooled with a large metal heatsink. I picked a really cool and very <em>blue</em> one from Geeekpi with a completely silent fan. Neat!</p>
<p>USB3/Gbit adapters are not all equivalent, there are good ones and really bad ones. I had excellent results with a TP-Link UE300 (11€ on Amazon in Oct 2021), where some other adapters I have never managed to get past 500Mbps. I picked the UE300 following a large number of recommendations on the OpenWRT forums. There were lots of doubts about the reliability of USB-based network adapters but so far I haven't had any trouble, the thing is remarkably stable.</p>
<p>Here are stats obtained after a couple of days of uptime:</p>
<pre class="literal-block">
eth1 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet addr:192.168.x.x Bcast:192.168.x.x Mask:255.255.255.0
inet6 addr: fe80::xxxx:xxxx:xxxx:xxxx/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:124584287 errors:0 dropped:125585 overruns:0 frame:0
TX packets:92614690 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:148704416920 (138.4 GiB) TX bytes:92963352990 (86.5 GiB)
</pre>
<p>This is the USB/Gbit NIC: no RX/TX errors. The dropped packets are due to the firewall doing its job.</p>
<p>Optionally: you may want to keep a USB keyboard and a screen with micro-HDMI plug handy. One of the advantages of using a Pi instead of a dedicated router is that it is meant to be a desktop, with two HDMI and additional USB ports. Compared to debugging a box with a JTAG link this is luxurious.</p>
<p>You probably want to keep a laptop or computer nearby with Ethernet cable to connect to the Pi when it is network-ready, though you could theoretically do everything from the command line on the Pi console.</p>
</div>
<div class="section" id="preparing-the-pi">
<h2>Preparing the Pi</h2>
<p>Optional: use a USB thumb drive to boot the Pi. You can of course use an SD card, but USB thumb drives seem a bit more reliable. The minimal size you want should be 1GB. To update the Pi's bootloader and EEPROM, follow the instructions from <a class="reference external" href="https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#raspberry-pi-4-boot-eeprom">the raspberrypi.org web site</a></p>
<p>You will need to flash a recent version of Raspberry OS on an SD card, boot from that, then call the <tt class="docutils literal"><span class="pre">raspi-config</span></tt> utility and make sure the EEPROM and bootloader are up-to-date, then switch the boot order to get a chance to boot from USB.</p>
<p>Next: you need the latest OpenWRT image for Raspberry Pi 4. Right now (Oct 2021) this is at:</p>
<div class="line-block">
<div class="line"><a class="reference external" href="https://downloads.openwrt.org/releases/21.02.0/targets/bcm27xx/bcm2711/">https://downloads.openwrt.org/releases/21.02.0/targets/bcm27xx/bcm2711/</a></div>
</div>
<p>You want <tt class="docutils literal"><span class="pre">rpi-ext4-factory-img.gz</span></tt></p>
<p>Alternatively: use OpenWRT image builder.</p>
<p>The next operations can be performed directly from your RPi4 as booted into Raspberry OS, but if you have a Linux machine handy it is probably best to use it.</p>
<p>Download the image, gunzip it, flash it to your USB thumb drive. You now have two partitions on your target USB thumb drive: first one is the DOS partition used for booting, the second one is an ext4 containing the whole OpenWRT system.</p>
<p>You want to resize the partitions as OpenWRT won't do that by itself. Use <tt class="docutils literal">parted</tt> or equivalent to:</p>
<ul class="simple">
<li>Resize the second partition to 1GB</li>
<li>Add a third partition to cover the rest of your USB drive</li>
<li>Resize the ext4 filesystem on the second partition with <tt class="docutils literal">resize2fs</tt></li>
<li>Create an ext4 filesystem on the third partition with <tt class="docutils literal">mkfs.ext4</tt></li>
</ul>
</div>
<div class="section" id="first-boot">
<h2>First Boot</h2>
<p>Your USB drive is now ready to boot! Plug it into the RPi4, remove the SD card, connect the screen and keyboard. Don't connect the USB/Gbit adapter yet, connect the Pi's native Ethernet to something that gives it Internet access as it will be needed to download the USB/Gbit driver. Power on the Pi. Your first login doesn't require a password and you are root.</p>
<p>Check you have Internet connectivity. If you don't it probably means the Pi's native Ethernet is registered for LAN usage. You can edit <tt class="docutils literal">/etc/config/network</tt> to temporarily switch it to DHCP like this:</p>
<pre class="literal-block">
config interface 'lan'
option device 'eth0'
option proto 'dhcp'
</pre>
<p>Restart the network with <tt class="docutils literal">/etc/init.d/network restart</tt> and shut down the firewall with <tt class="docutils literal">/etc/init.d/firewall stop</tt>.</p>
<p>Once online, update the list of packages:</p>
<pre class="literal-block">
opkg update
</pre>
<p>Download the driver for your USB/Gbit adapter. For UE300 this is <tt class="docutils literal"><span class="pre">kmod-usb-net-rtl8152</span></tt>:</p>
<pre class="literal-block">
opkg install kmod-usb-net-rtl8152
</pre>
<p>Installing the driver won't show you an additional Ethernet device yet. You need to edit <tt class="docutils literal">/etc/config/network</tt> once more to assign your NICs. I chose to keep <tt class="docutils literal">eth0</tt> (the native Ethernet) for LAN and use <tt class="docutils literal">eth1</tt> (the USB dongle) for WAN. Pick a static address for your router: in the following example I chose 192.168.44.1:</p>
<pre class="literal-block">
config interface 'loopback'
option device 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config interface 'lan'
option device 'eth0'
option proto 'static'
option ipaddr '192.168.44.1'
option netmask '255.255.255.0'
config interface 'wan'
option device 'eth1'
option proto 'dhcp'
</pre>
<p>Restart the network and you should be able to see all interfaces with <tt class="docutils literal">iconfig</tt>.</p>
</div>
<div class="section" id="configure-openwrt">
<h2>Configure OpenWRT</h2>
<p>You now have a working RPi4 router but can't do much with it yet. Use an Ethernet cable to connect a laptop/desktop to the Pi's LAN, in my case this is <tt class="docutils literal">eth0</tt>, the Pi's native Ethernet. Manually assign an IP on that link to be part of the same subnet, e.g. 192.168.44.100, make sure your link is up, open a browser, browse to URL <tt class="docutils literal"><span class="pre">http://192.168.44.1/</span></tt></p>
<p>You should get the LuCi interface for OpenWRT. Log in as root (no password) and from there on you are in OpenWRT territory. Time to read the docs, understand it a bit better, and configure it to suit your needs.</p>
<p>Next steps:</p>
<div class="section" id="mount-your-third-partition">
<h3>Mount your third partition</h3>
<p>You prepared a third partition on the USB thumb drive, you can mount it for additional storage space e.g. on <tt class="docutils literal">/data</tt>:</p>
<pre class="literal-block">
opkg install block-mount
block detect > /etc/config/fstab
</pre>
<p>Edit <tt class="docutils literal">/etc/config/fstab</tt> and keep only the last entry, the one for your third partition. The result should look like:</p>
<pre class="literal-block">
config mount
option target '/data'
option uuid 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
option options 'noatime'
option enabled '1'
</pre>
<p>where the uuid corresponds to your third partition, as detected by <tt class="docutils literal">block detect</tt>.</p>
</div>
<div class="section" id="make-it-more-unix-like">
<h3>Make it more Unix-like</h3>
<p>Remember to run <tt class="docutils literal">opkg update</tt> before you start installing stuff.
You probably want to add a non-root user and some default Unix utilities like bash, vim, tmux, and diff:</p>
<pre class="literal-block">
opkg update
opkg install bash vim tmux diffutils
opkg install shadow-useradd shadow-groupadd
</pre>
<p>Do <em>not</em> install <tt class="docutils literal"><span class="pre">shadow-passwd</span></tt> as it will completely mess up your password file.</p>
<p>Use <tt class="docutils literal">useradd</tt> and <tt class="docutils literal">groupadd</tt> to add a non-root user with bash shell and home in /home. Give it a password with <tt class="docutils literal">passwd USERNAME</tt>.</p>
<p>Try login in as that user now and make sure you can <tt class="docutils literal">su</tt> to root. You can also change root's default shell by editing <tt class="docutils literal">/etc/passwd</tt>. You can also add <tt class="docutils literal">sudo</tt>, don't forget to add your non-root users to the list of sudoers.</p>
</div>
<div class="section" id="add-networking-tools">
<h3>Add networking tools</h3>
<p>Add some utilities:</p>
<pre class="literal-block">
opkg install iperf3 tcpdump netcat nmap ethtool
</pre>
<p>iperf3 is always useful for benchmarking from the router itself. tcpdump, netcat, nmap, and ethtool are convenient tools to have handy on a router.</p>
</div>
<div class="section" id="add-statistics">
<h3>Add Statistics</h3>
<p>Additional services and their interfaces:</p>
<ul class="simple">
<li><tt class="docutils literal">nlbwmon</tt>, <tt class="docutils literal"><span class="pre">luci-app-nlbwmon</span></tt>: bandwidth monitoring tools with details per device</li>
<li><tt class="docutils literal">vnstat</tt>, <tt class="docutils literal"><span class="pre">luci-app-vnstat</span></tt>: bandwidth daily, weekly, monthly usage with graphs</li>
<li><tt class="docutils literal"><span class="pre">luci-app-statistics</span></tt> for more general statistics about your router</li>
<li><tt class="docutils literal"><span class="pre">luci-app-wol</span></tt> for wake-on-lan support on the web interface</li>
</ul>
</div>
<div class="section" id="replace-dropbear-with-openssh">
<h3>Replace dropbear with openssh</h3>
<p>The default ssh server on OpenWRT is dropbear. You can replace it with openssh to make things a bit more familiar. Dropbear does not support as many ssh key types (e.g. ed25519) and centralizes all trusted keys inside <tt class="docutils literal">/etc/dropbear</tt>. Since you have a non-root user you probably want to manage its keys in <tt class="docutils literal"><span class="pre">~/.ssh</span></tt> instead.</p>
<p>Open a first ssh connection to your device and leave it open. Open a second one and move dropbear to listen to another port instead, like 2222. Edit <tt class="docutils literal">/etc/config/dropbear</tt>:</p>
<pre class="literal-block">
config dropbear
option PasswordAuth 'on'
option RootPasswordAuth 'on'
option Port '2222'
</pre>
<p>Restart it with <tt class="docutils literal">/etc/init.d/dropbear restart</tt> and make sure you can still connect to your router:</p>
<pre class="literal-block">
ssh -p 2222 root@192.168.44.1
</pre>
<p>Now install openssh and related packages and start it up:</p>
<pre class="literal-block">
opkg install openssh-server openssh-sftp-server
opkg install openssh-client openssh-keygen
/etc/init.d/sshd start
</pre>
</div>
</div>
<div class="section" id="backing-up-vnstat">
<h2>Backing up vnstat</h2>
<p><tt class="docutils literal">vnstat</tt> forgets everything on each boot. You probably want to persist those stats as they may be useful to monitor the long-term health of your router. You can find a nice description of how to achieve that here:</p>
<div class="line-block">
<div class="line"><a class="reference external" href="https://openwrt.org/docs/guide-user/services/network_monitoring/vnstat">https://openwrt.org/docs/guide-user/services/network_monitoring/vnstat</a></div>
</div>
</div>
<div class="section" id="monitoring-temperatures">
<h2>Monitoring Temperatures</h2>
<p>OpenWRT is a Linux distribution so you get some perks too: the CPU temperature can be read from the pseudo-file <tt class="docutils literal">/sys/class/thermal/thermal_zone0/temp</tt> which gives you a temperature in milliCelsius. To get those readings into reported statistics I had to mess up with <tt class="docutils literal">/etc/config/collectd</tt> and add that section:</p>
<pre class="literal-block">
config plugin 'thermal'
option enable '0'
option IgnoreSelected '0'
option ForceUseProcfs 'false'
</pre>
</div>
<div class="section" id="adding-static-dhcp-leases">
<h2>Adding static DHCP leases</h2>
<p>For all machines on my home network I like to assign static IP addresses. Two ways to achieve that:</p>
<p>Edit /etc/config/dhcp and add one section per machine, like:</p>
<pre class="literal-block">
config host
option name 'myprinter'
option mac '00:00:00:f4:29:ad'
option ip '192.168.44.20'
</pre>
<p>Or use a script to generate two files: <tt class="docutils literal">/etc/hosts</tt> and <tt class="docutils literal">/etc/ethers</tt>, containing respectively:</p>
<p>The list of IP/name pairs:</p>
<pre class="literal-block">
# Hosts
192.168.44.1 router
192.168.44.2 wifi
# etc.
</pre>
<p>and the list of MAC/name pairs:</p>
<pre class="literal-block">
00:00:00:00:20:e2 router
00:00:00:aa:20:5e wifi
</pre>
<p><tt class="docutils literal">dnsmasq</tt> will honour the presence of those two files if you select the right options in Services/DHCP.</p>
</div>
<div class="section" id="adding-an-ad-blocker">
<h2>Adding an Ad Blocker</h2>
<p>Adding a DNS-based ad blocker is probably a good idea. Check out this write-up for a good description of how to install AdGuardHome on a RPi4/OpenWRT box:</p>
<div class="line-block">
<div class="line"><a class="reference external" href="https://bayas.dev/posts/adguard-home-openwrt/">https://bayas.dev/posts/adguard-home-openwrt/</a></div>
</div>
<p>The latest version I installed with opkg (<tt class="docutils literal">opkg install adguardhome</tt>) comes with an incompatible binary. Just download the arm64 version from the AdGuardHome github, replace it in <tt class="docutils literal">/usr/bin</tt> and you should be all set.</p>
</div>
<div class="section" id="gallery">
<h2>Gallery</h2>
<p>Now for some screenshots just to spice up that post.</p>
<p>The main status page:</p>
<img alt="Status page" src="images/openwrt-01-status.png" />
<p>CPU usage for the first two cores is fairly limited. The last two cores are mostly unused.</p>
<img alt="CPU usage" src="images/openwrt-05-stats1.png" />
<p>Network usage on the internal NIC:</p>
<img alt="Network usage" src="images/openwrt-06-stats2.png" />
<p>Temperature monitoring:</p>
<img alt="Temperature" src="images/openwrt-07-stats3.png" />
<p>Memory usage is just ridiculous. About 4 gigs free out of 4 gigs. Try finding the smallest RPi4 you can lay your hands on.</p>
<img alt="Memory usage" src="images/openwrt-08-stats4.png" />
</div>
<div class="section" id="conclusion">
<h2>Conclusion</h2>
<p>Give it a try! Worst that could happen: your RPi4 goes back to gathering dust on a shelf with the other Pis.</p>
</div>
Home Router Experiments2021-10-03T22:00:00+02:002021-10-03T22:00:00+02:00nicolas314tag:maz.one,2021-10-03:/home-router-experiments.html<p>Latest upgrade on my home Internet connection brought me to an almost symmetric Gbit link: 1000 Mbit/s down, 500 Mbit/s up. Very comfortable, but what kind of home router do you need to sustain such traffic loads? I tried a lot of alternatives which I will briefly describe …</p><p>Latest upgrade on my home Internet connection brought me to an almost symmetric Gbit link: 1000 Mbit/s down, 500 Mbit/s up. Very comfortable, but what kind of home router do you need to sustain such traffic loads? I tried a lot of alternatives which I will briefly describe below. The final outcome was quite surprising. Read on for more.</p>
<div class="section" id="what-does-gbit-mean-anyway">
<h2>What does Gbit mean anyway?</h2>
<p>Gigabit Network Interface Cards (NICs) are theoretically capable of pumping 1 Gbit/s in both directions simultaneously, reaching an actual total of 2 Gbit/s combined. That's a lot of things to perform for a GHz-class CPU. As a rule of thumb, you cannot usually reach such speeds with embedded sub-GHz CPUs without specific hardware acceleration.</p>
<p>For more information on this topic and a very worthwhile discussion, read on here:</p>
<p><a class="reference external" href="https://forum.openwrt.org/t/so-you-have-500mbps-1gbps-fiber-and-need-a-router-read-this-first/90305">https://forum.openwrt.org/t/so-you-have-500mbps-1gbps-fiber-and-need-a-router-read-this-first/90305</a></p>
</div>
<div class="section" id="software-first">
<h2>Software first</h2>
<p>There aren't so many home router software bundles, so with a bit of patience you could try them all and see what they're worth:</p>
<ul class="simple">
<li>pfSense, OPNsense are based on FreeBSD</li>
<li>IPFire is open-source and free, Linux-based</li>
<li>Untangle, ClearOS, Sophos are commercial, Linux-based</li>
<li>OpenWRT: free and Linux-based</li>
<li>VyOS: free/commercial and Linux-based</li>
</ul>
<p>Worth noting: a whole lot of experiments based on FreeBSD, OpenBSD, and Linux.</p>
<p>Check Wikipedia for more ideas:
<a class="reference external" href="https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributions">https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributions</a></p>
<p>You can also roll your own with a straight OS install:</p>
<ul class="simple">
<li>FreeBSD with pf or ipfw</li>
<li>OpenBSD with pf</li>
<li>Debian or Arch with Shorewall or nftables</li>
</ul>
<p>In no particular order and without any scientific or systematic method, here are my findings:</p>
</div>
<div class="section" id="ubiquiti-edgerouter-x">
<h2>Ubiquiti EdgeRouter-X</h2>
<p><a class="reference external" href="https://store.ui.com/collections/operator-edgemax-routers/products/edgerouter-x">https://store.ui.com/collections/operator-edgemax-routers/products/edgerouter-x</a></p>
<p>This cheap ($50) box comes pre-installed with Ubiquiti's EdgeOS, a Debian-based OS that still retains some compatibility as you can install some Debian packages through CLI. I wasn't much impressed with EdgeOS: the GUI is nice but hard to browse through. Adding a static DHCP lease forces you to click-click-click far too much, especially when you want to enter a hundred MAC and IP addresses. I ended up doing most things through ssh. Bonus points: you can open the box and directly access the JTAG for serial console fun, or re-flashing the firmware.</p>
<p>OpenWRT can be flashed on this tiny box. The results are excellent: all the comfort of OpenWRT's Luci GUI and the power of a true Linux CLI. Only trouble is the performance: I managed to get a window of about 900 Mbit/s in total, so about 450-450 up-down which is still far from maxing my connection. If your Internet bandwidth is below that you shouldn't hesitate a single moment and go for it, you won't find any better.</p>
<p><a class="reference external" href="https://forum.openwrt.org/t/ubiquiti-edgerouter-x-loading-openwrt-and-performance-numbers/27470">https://forum.openwrt.org/t/ubiquiti-edgerouter-x-loading-openwrt-and-performance-numbers/27470</a></p>
<p>I realize I could be running my home network on this tiny box and wouldn't notice any difference since I pretty much <em>never</em> use 1000+500 simultaneously, but the real fun is to manage to squeeze out every single bit/s from my line, because I can.</p>
</div>
<div class="section" id="mikrotik-hex-s">
<h2>Mikrotik hEX S</h2>
<p><a class="reference external" href="https://mikrotik.com/product/hex_s">https://mikrotik.com/product/hex_s</a></p>
<p>This box has the very same hardware as Ubiquiti's EdgeRouter-X, just sold with a different proprietary Linux-based firewall. Bit pricier too: $70 at the time of writing.</p>
<p>The firewall GUI looks outdated, like a web site from the 1990s that hasn't aged well. The left-side menu is just a long list of confusing acronyms with not even an attempt at explaining what they do. It takes ages reading the copious online documentation to just configure a basic NAT and a few rules. At that level I am wondering if this GUI is meant to set up a firewall or just torture humans.</p>
<p>I spent a lot of time on Reddit looking for clues and recommendations. Seems you can only love or hate that GUI, so count me with the haters on that one. I sent the hardware back to Amazon after a couple of weeks.</p>
<p>Good news: you can re-flash that box with OpenWRT! Bad news: the box is only spec'd for about 900 Mbit/s total with a few firewall rules and a NAT. Nice but won't saturate my line, and I cannot condone torture through GUI.</p>
<p><a class="reference external" href="https://mikrotik.com/product/hex_s#fndtn-testresults">https://mikrotik.com/product/hex_s#fndtn-testresults</a></p>
</div>
<div class="section" id="pcengines-apu2c4">
<h2>PCengines APU2C4</h2>
<p><a class="reference external" href="https://pcengines.ch/apu2c4.htm">https://pcengines.ch/apu2c4.htm</a></p>
<p>That box is touted everywhere as the best one in its price range. For 100-150 euros you get 3 Intel NICs attached to an AMD quad-core processor. The part I like most is the open-source BIOS (coreboot) which keeps improving all the time. My poor box has been reflashed a million times to try all possible software options. In details:</p>
<ul class="simple">
<li>pfSense worked, but not great. Best speed I could reach was about 600 Mbit/s download. That was a while ago though. TekLager published several recommendations to squeeze better performance out of pfSense, which I haven't tried.</li>
</ul>
<p><a class="reference external" href="https://teklager.se/en/knowledge-base/apu2-1-gigabit-throughput-pfsense/">https://teklager.se/en/knowledge-base/apu2-1-gigabit-throughput-pfsense/</a></p>
<ul class="simple">
<li>OPNsense did just about the same as pfSense. The GUI is a lot more friendly and there are more functionalities offered, but the engine running underneath reached exactly the same disappointing throughput. Again: haven't tried in a while, possibly performs better today.</li>
<li>FreeBSD+pf did a bit better, reaching 700 Mbit/s download. That was FreeBSD 11 and I had to tweak a lot of stuff to make it work. Most certainly better on the latest FreeBSD 13, but I haven't tried. I couldn't find any sexy GUI to display graphs or just a dashboard for a quick glance at my network status.</li>
<li>OpenBSD+pf got the worst results, painfully reaching 300 Mbit/s download. Activating multi-processor support brought it up to 400 Mbit/s and the box got really warm during stress tests. There is of course no GUI but the pf configuration syntax is really clever and easy to write. The pf book is a fantastic source for learning how to build your own firewall. Even if the performance was really disappointing I learned so much about OpenBSD and firewalling that it was in itself a very worthwhile experiment. Would do it again just for fun.</li>
</ul>
<p>If you are interested in building an OpenBSD firewall, here is a good tutorial:
<a class="reference external" href="https://openbsdrouterguide.net/">https://openbsdrouterguide.net/</a></p>
<ul class="simple">
<li>OpenWRT works fine on this box, though still not reaching symmetric Gbit. The latest OpenWRT version I tried (2021.02.0) seems to have a bug: I could only reach 300 Mbit/s download when upload had no trouble reaching my 500 Mbit/s limit. I remember getting far better results on OpenWRT 19.x versions.</li>
<li>Debian+Shorewall wasn't so good. Can't remember what speeds I reached but it was far from saturating the line. Shorewall is also a beautiful piece of software for configuring a firewall, with adequate documentation and examples. The project seems to have been orphaned in 2018 and is still very useful and usable today, just don't expect new features.</li>
</ul>
<p>I also tried Debian/nftables, with same disappointing performance. I didn't know anything about nftables when I started and read all available documentation. About 90% of the docs I found were meant to let me know why nftables is soooo much better than iptables and will end up replacing it one day, but only 10% are dedicated to explaining how to actually build a firewall, mostly by translating iptables rules to nftables. Since I didn't know about iptables either it didn't help me much. The syntax looks nice though, similar to BSD's pf with a few more warts. I wish they had written a manual.</p>
<ul class="simple">
<li>IPFire got me in trouble before I could use it. I managed to mess up the internal network config during install and couldn't recover from command-line afterwards as everything is meant to be controlled from the web interface. I was so annoyed I just gave up and didn't even try it out, though they claim to reach full symmetric Gbit on that hardware. Maybe another day.</li>
</ul>
<p>I also ran a few experiments with Proxmox and installed virtual instances of OpenWRT and OPNsense with direct PCI passthrough to the NICs. The box reached the same performance overall as their direct install counterparts in both cases. The only real advantage of virtualizing your router is you can snapshot it at regular intervals. Proxmox is a lot of fun to play with.</p>
</div>
<div class="section" id="raspberry-pi-4">
<h2>Raspberry Pi 4</h2>
<p>Spending a lot of time on OpenWRT forums, I stumbled upon many discussions about using an RPi4 as a home router for symmetric Gbit. It seemed hard to believe: those little critters are notoriously underpowered and have only one Ethernet port anyway, so what's the point?</p>
<p>Somebody tried one out with a USB3 Gbit dongle from various vendors and found out that they all yield very different performances. Most specifically: the TP-Link UE300 (a Realtek NIC for $11 on Amazon as I write this) seems to perform exceedingly well. This discovery was mostly met with disbelief at first but then some people started actually benchmarking the whole thing with great care and confirmed the outcome: an RPi4/UE300 combination running OpenWRT can sustain full symmetric Gbit without a sweat. Wow!</p>
<p>Of course I had a couple of RPi4 gathering dust somewhere at home. I just picked one up, swapped the SD card with a recent OpenWRT version, connected a TP-Link UE300 and started running tests. The results are just astonishing: full symmetric Gbit support on an internal network (941+ Mbit/s up and 941+ Mbit/s down). I have used one on my home network for a couple of weeks now and can confirm this is working astonishingly well.</p>
<p>I ended up bulletproofing my setup: got a box and a fan to keep it cool, a USB3 thumb drive to install the OS instead of running from a flaky SD card, and off you go. In terms of cost it ends up being twice as expensive as an EdgeRouter-X for double performance and it is so much fun to use I do not regret it a single moment.</p>
<p>More about that setup in another blog post.</p>
</div>
Self-hosting ideas2021-05-31T22:51:00+02:002021-05-31T22:51:00+02:00nicolas314tag:maz.one,2021-05-31:/self-hosting-ideas.html<img alt="self-hosting" src="/images/selfhosting.png" />
<p>Self-hosting is about running your own cloud services on your own hardware. Remember: there is no cloud, you are just using somebody else's computer. Sometimes it makes sense, sometimes you don't want to entrust that someone else with your data. In some cases it is a lot cheaper to host …</p><img alt="self-hosting" src="/images/selfhosting.png" />
<p>Self-hosting is about running your own cloud services on your own hardware. Remember: there is no cloud, you are just using somebody else's computer. Sometimes it makes sense, sometimes you don't want to entrust that someone else with your data. In some cases it is a lot cheaper to host a service yourself than delegate it online, like hoarding terabytes of media files. Self-hosting is also a good way to learn the latest fashions: Virtual Machines, containers, and how to automate everything around those.</p>
<p>The downside is that you have to manage everything yourself and your uptime is unlikely to match what is offered by cloud service providers. If you are your only customer that should be fine, but if you end up managing the service for multiple users you will need to figure out what to do when you are not interested any more or if you get run over by a bus. Size your risks and make the right decision.</p>
<p>The self-hosting community is fairly active on Reddit. Check out those two subreddits if you haven't yet:</p>
<div class="line-block">
<div class="line"><a class="reference external" href="https://www.reddit.com/r/selfhosted/">https://www.reddit.com/r/selfhosted/</a></div>
<div class="line"><a class="reference external" href="https://www.reddit.com/r/DataHoarder/">https://www.reddit.com/r/DataHoarder/</a></div>
</div>
<p>If you are looking for ideas about what to self-host, this curated list has everything you want:</p>
<p><a class="reference external" href="https://github.com/awesome-selfhosted/awesome-selfhosted">https://github.com/awesome-selfhosted/awesome-selfhosted</a></p>
<p>Just for fun, here are some self-hosted services I have either used or contemplated using.</p>
<p><a class="reference external" href="https://bitwarden.com/">Bitwarden</a></p>
<p>Bitwarden is a commercial service offering storage for your online passwords. There are a lot of similar services but this one has a perfectly usable free tier. You also have the possibility to <a class="reference external" href="https://github.com/dani-garcia/vaultwarden">host your own server</a>, so your passwords are never stored on anybody else's computer. That latter is not officially supported by Bitwarden but apparently fully operational, though I haven't tried it myself.</p>
<p>Best thing about Bitwarden is that you have browser extensions and mobile apps so it is actually usable where you need it.</p>
<p><strong>A Dropbox replacement</strong></p>
<p>The <a class="reference external" href="https://www.seafile.com/">Seafile</a> experience is very similar to dropbox. Install it on a server and give it its own disk space, then access it through a web interface or one of the many clients (including mobile). Seafile has its own storage system, files are not mapped 1:1 to a local filesystem but stored inside containers that can only be unwrapped using a seafile client -- though they have a seafile-fuse utility for mounting a seafile container as a local directory.</p>
<p>Advantages: Seafile is really fast compared to Owncloud/Nextcloud, but not meant to store large collections of gigabyte-sized files. Useful when sharing PDFs or similar. The Seafile mobile client is nice.</p>
<p>I had a lot of trouble with the brittle install procedure. As soon as you deviate from defaults it tends to crash in mysterious ways and your best bet is to restart the procedure again. But once installed, it works great! Just don't stray off the defaults.</p>
<p><a class="reference external" href="https://nextcloud.com/">Nextcloud</a> and Owncloud are two similar projects aiming at providing a full solution for file-sharing and communication in teams. Useful to share files among a small group. I only tried Nextcloud.</p>
<p>Installing Nextcloud is a major pain in the butt. I could not manage to get it to a working state with either docker images or a freshly installed virtual machine. The only thing that worked for me was installing dietpi on a Raspberry Pi and running the automated Nextcloud install from there, but I will admit having very little patience with those things.</p>
<p>My biggest gripe with Nextcloud is the sheer amount of software it relies on, including an Apache/PHP combination I would rather avoid. Nextcloud also requires a database engine to work. Though it can technically be satisfied with an Sqlite3 database, it keeps moaning that you won't have a true Nextcloud experience until you install MySQL or Postgres. My experience running it on a Raspberry Pi 4 was not fantastic as the interface is rather slow. The mobile app had trouble importing photos, ending up with a large number of duplicates all over the place. Your mileage may vary.</p>
<p>The main advantage of Nextcloud is the large community around it.</p>
<p>On the topic of file synchronisation: <a class="reference external" href="https://syncthing.net/">Syncthing</a> is a portable solution that will maintain a data directory identical on all machines connected to the same account. Add/modify/delete a file on one machine and the status will be replicated everywhere. It is extremely convenient for effortless backups.</p>
<p>Syncthing does not work instantaneously though. Synchronisation is based on torrents that can take a while to start, so do not expect to use it for collaborative tasks between users. I prefer rsync for my own needs because I like to know when directories get synchronised, but Syncthing offers an excellent hands-free alternative to dropbox.</p>
<p><a class="reference external" href="https://github.com/filebrowser/filebrowser">filebrowser</a></p>
<p>If you are looking for a much simpler alternative to Seafile/Nextcloud, look no further! <a class="reference external" href="https://github.com/filebrowser/filebrowser">filebrowser</a> is a Golang single-file executable that requires no Apache/PHP, no database engine, and runs in a single process. Place your carefully sorted files into a directory and let filebrowser do the butler service for you. There are no client apps, just a web-based interface that works extremely fast.</p>
<p>You can upload/download files or entire directories, preview media files, but mostly: you can manage your data directly on the filesystem. That piece of software runs on every single machine I use to store data. If all you care about is exposing a filesystem on a web interface for easy file management, go for it.</p>
<p><a class="reference external" href="https://github.com/navidrome/navidrome">Navidrome</a></p>
<p>I have accumulated a large collection of music files over the years and finding the right kind of tool to access them has always proved to be difficult. Most programs will choke at the time of indexing a large media collection. I remember letting KDE's Amarok run for several days and finally giving up with some weird error message, but that was years ago. Things surely improved since then.</p>
<p>A rather recent project called <a class="reference external" href="https://github.com/navidrome/navidrome">navidrome</a> seems to have the indexing issue in hand, and offers itself through a really fantastic web interface. Navidrome is a Golang single-file executable: point it to your music collection, point a browser to the main page, done. Your albums will start showing up immediately as they are indexed. No need to run a separate database, dedicated web server or PHP interpreter, everything works out of the box.</p>
<p>The only inconvenient is that you are only browsing your own music files. If you want to discover new talents, nothing replaces the usual music services we have today. Having navidrome gives me access to all of my music online but I still maintain my subscription to a music provider because it is so easy to use, available everywhere, and points me to artists I never heard about.</p>
<p><strong>Media servers</strong></p>
<div class="line-block">
<div class="line">So you have a large collection of movies and want to watch them? <a class="reference external" href="https://www.plex.tv/">Plex</a> seems to be the most popular choice today, offering a Netflix-like interface to your own media. Plex comes with a number of mobile apps in addition to a very nice web interface.</div>
<div class="line">The downside is that Plex runs on a subscription, which means you have to log onto the Plex server to use your own instance. That's silly: if your Internet connection is down you won't be able to browse the movies you have in front of you in that media box. They have a free tier but I just can't accept the fact I have to be connected to a remote server to watch my own local media.</div>
</div>
<p>Last time I tried Plex (free tier), I was greeted with a screenful of ads for TV shows and programs that made it hard to find my own local content. Not really my cup of tea. Seems that Plex has become an ad distribution program with the ability to watch your own movies.</p>
<p>Alternatives to Plex are <a class="reference external" href="https://emby.media/">emby</a> or <a class="reference external" href="https://jellyfin.org/">jellyfin</a>. Emby started its life as an open-source project and is now a (mostly) proprietary system attached to a company, like Plex. Jellyfin is an open-source project that looks like Plex or Emby without the ads. The web interface is not as nice and the Android client I tried had some bugs, but overall it mostly works and looks very promising. Give it a bit more time and it should become a true replacement for Plex.</p>
<p>If all you want is access to your local media, install <a class="reference external" href="https://kodi.tv/">Kodi</a> and be done with it. Kodi works great on local media, even when they are stored on a Samba share. There are tons of interesting plugins to watch Internet TV if that's your thing.</p>
<p>Careful though: Kodi can be unstable. The latest major update broke a large number of plugins and skins, making it unusable for a few days. If all you want is browse a directory and watch movies, VLC also works very well on Android TV and any Linux box.</p>
<p><strong>Minecraft</strong></p>
<p>Setting up a Minecraft server for the kids was a blessing during those long lockdown days. Careful though: there are currently two versions of Minecraft. If you install the <a class="reference external" href="https://www.minecraft.net/en-us/download/server">java-based Minecraft server</a>, players who have the Microsoft version of that game won't be able to join. Make sure you read the docs before you purchase.</p>
<p><strong>Source Control</strong></p>
<div class="line-block">
<div class="line">A lot of self-hosters have their own github clone, based on something like <a class="reference external" href="https://gitea.com/">Gitea</a>.</div>
<div class="line">If all you are looking for is source control for your own software, you should definitely give <a class="reference external" href="https://fossil-scm.org">Fossil</a> a try. This version control system is nothing short of amazing! Written by the same author who gave the world Sqlite, Fossil includes everything you need to support software projects: version control with branching and merging à la git, ticketing system, wiki, and notes. All of your project data are contained into a single file: an Sqlite database you can rsync around for backups.</div>
</div>
<p>I use it myself to document my various personal projects, keep notes handy, or store tidbits of information in a personal wiki. Fossil is a single executable programmed in C.</p>
<p><strong>Online Book Storage</strong></p>
<p>How do you browse your large collection of epub files? <a class="reference external" href="https://calibre-ebook.com/">Calibre</a> seems to be the most popular desktop solution, but it can be completed with <a class="reference external" href="https://github.com/janeczku/calibre-web">calibre-web</a>, an independent project that will read your Calibre library and make it available through a web interface. That allows you to read books directly on your web browser, including on mobile devices.</p>
<p>If you are into comics, CBR/CBZ files can be served by <a class="reference external" href="https://vaemendis.net/ubooquity/">ubooquity</a>, an easily-installed solution that offers directories full of comics on a web interface, saving the hassle of having to copy sometimes large files onto your tablet for reading.</p>
<p>There are many more things you could self-host if you give yourself some time to do it. Stay in control of your data!</p>
Being reachable on a public IP address2021-05-23T19:09:00+02:002021-05-23T19:09:00+02:00nicolas314tag:maz.one,2021-05-23:/being-reachable-on-a-public-ip-address.html<p><em>Note: I am not affiliated or employed by any of the providers mentioned below. I am only providing links for convenience, I make no money or karma when you click any of those.</em></p>
<img alt="selfhosting" src="/images/selfhosting.png" />
<p>Say you want to host some service at home and make it reachable from the Internet. What …</p><p><em>Note: I am not affiliated or employed by any of the providers mentioned below. I am only providing links for convenience, I make no money or karma when you click any of those.</em></p>
<img alt="selfhosting" src="/images/selfhosting.png" />
<p>Say you want to host some service at home and make it reachable from the Internet. What are your options?</p>
<p>Most obvious choice: start a server on your home network, forward the adequate ports from your home router, done. While this is by far the easiest solution, it is not ideal for several reasons:</p>
<ul class="simple">
<li>If your home router belongs to your ISP, it may not have the option to forward ports to your internal network.</li>
<li>Even with your own router, your ISP may simply not allow open the ports you want to the outside world. I have experience with an ISP using ports 80 and 443 for their own router needs, making it impossible to use them for home services.</li>
</ul>
<p>If the service you want to host at home uses exotic port numbers like a Synology NAS or a NextCloud instance, it should be fine. Just forget about hosting your own web site at home.</p>
<p>Your next hurdle might simply be your public IP address. In many cases, ISPs provide temporary public IP addresses. That should not be a problem with a little help from a dynamic DNS provider. Open an account with a provider like <a class="reference external" href="https://www.noip.com/">no-ip.com</a> or <a class="reference external" href="https://account.dyn.com/">dyndns</a> and make sure you run their client on a 24x7 server on your home network to update your dynamic DNS entry every time your public IP changes.</p>
<p>That is exactly what I did in the past. I used dyndns for a while, then switched to no-ip when dyndns became a pain to use. In all cases you get a choice of the subdomain and your server ends up being something like <em>myserver.yet-another-dynamic-provider.com</em>. That's fine if all you want is a pointer to your home server but if you are serving web pages or anything relevant to a larger audience than yourself, those names are not really cool.</p>
<p>Another option would be to purchase a domain name from a registrar that offers an API (ideally an SDK) to allow you to script a name/address change from your home server. Domain names can get really cheap those days, especially for top-level domains like .xyz. Don't get attracted by the latest discounts, the important price to watch is the renewal price. I have rented some names for less than $1/year the first year, only to be greeted with a $30+/year renewal price a year later when I had started circulating that domain name to everyone who should care about it. That's not cool.</p>
<p>Once you have your own domain name, the procedure to follow is to run a cron job every minute or so to do:</p>
<ul class="simple">
<li>Get the IP address associated to <em>myservice.mydomain</em></li>
<li>Get your current home public IP address</li>
<li>If they are different, call the script to update <em>myservice.mydomain</em> with your new public IP address. Don't forget to set a short TTL for that particular name so it hopefully gets updated fast when the address changes.</li>
</ul>
<p>Updating your IP/name should ideally be delegated to your home router if you could reliably program it to trigger an event every time your public IP changes, together with the new address. Unfortunately most ISP-provided routers are not programmable. Dynamic DNS providers will give you a client to download that does just what is described above, running on a separate machine on your home network.</p>
<p>A reliable way to learn your own public IP address is to ask a service like <a class="reference external" href="https://ipinfo.io/">ipinfo.io</a>.</p>
<p>Something as simple as:</p>
<pre class="literal-block">
curl ipinfo.io
</pre>
<p>There are several sites available for that kind of service and most of them have rate-limiting so make sure your client update script is not started too often, depending on terms and conditions from the site you use.</p>
<p>With that in hand you should be able to make your home server visible from the Internet on a reachable name. You may still suffer from ports like HTTP/HTTPS being blocked on your router, forcing you to publish an address for your self-hosted blog that look like <em>myblog.mydomain:9998/</em></p>
<p>That works but that isn't very nice. It opens a port on your home IP address, which widens your attack surface. If the service you host gets caught into a DDoS storm you won't be able to use your Internet connection until the storm subsides.</p>
<p>After years of using such tricks, I moved on to another solution that involves renting an IP address and a server. Let me elaborate:</p>
<p>Go find a cheap Virtual Private Server (VPS) online. In Europe you can probably rent one from <a class="reference external" href="https://www.ovhcloud.com/en/vps/">OVH</a> or <a class="reference external" href="https://www.hetzner.com/cloud">Hetzner</a> for a handful of euros per month. You don't care about powerful CPUs, lots of RAM, or lots of disk space. All you want is enough power to run a minimal Linux distro, a public IP address, and ideally unlimited traffic on a reasonable bandwidth. The cheapest I could find is 2 euros/month for unlimited traffic with 250Mbit/s bandwidth.</p>
<p>That VPS will be your front-end to the Internet. Register its IP address with your DNS like myblog.mydomain and this shouldn't change until you decide to move to another VPS. No need to run an update script every minute.</p>
<p>Next step is to create a permanent link between your home network and your VPS. Several options there:</p>
<ul class="simple">
<li>Run a reverse SSH tunnel if you know how this works, e.g. with <a class="reference external" href="https://man.archlinux.org/man/autossh.1">autossh</a>.</li>
<li>Run your own Virtual Private Network (VPN) between your VPS and your home network.</li>
</ul>
<p>I won't get into how to use autossh to build a tunnel that automatically restarts when the link gets broken as there are excellent tutorials for that. You absolutely need to plan for that link to get broken every time your home Internet connection gets interrupted, e.g. when someone brutally unplugs your router to plug in the vacuum cleaner, or when your ISP decides to upgrade your modem firmware. Get real: your home connection is not in a data center.</p>
<p>There are several options for running your own VPN to your VPS:</p>
<p><strong>IPSec</strong></p>
<p>That stuff is reserved for network admins who received a complete training on setting it up and running it correctly. Don't try this at home, life is too short.</p>
<p><a class="reference external" href="https://openvpn.net/">OpenVPN</a></p>
<p>I used that one for years until Wireguard became available on all recent Linux boxes. OpenVPN is a cool project but very heavy in terms of resources and not very performant. Configuring a simple point-to-point OpenVPN server is a major pain in the butt that is likely to take you several hours every time it stops working. You may be lucky enough to have an option to run an OpenVPN client on your home router (e.g. if you built your own with OpenWRT) but don't count on it. A GUI won't make your configuration issues magically go away.</p>
<p><a class="reference external" href="https://www.wireguard.com/">Wireguard</a></p>
<p>Wireguard is the new VPN kid on the block. This (entirely free) Linux kernel module is now included into most Linux distros and works out of the box with very few configuration steps. You can find Wireguard clients for Linux, Mac, Windows, Android, and iOS. In most cases the client deals directly with the OS kernel, making it very efficient and extremely fast compared to OpenVPN. Subscribing a mobile phone onto a VPN is just a matter of scanning a QR code, it is ridiculously simple. There are excellent tutorials about how to set up your own Wireguard VPN.</p>
<p><a class="reference external" href="https://tailscale.com/">Tailscale</a></p>
<p>Tailscale is a commercial service available for free to home users, within some reasonable limits. Tailscale is based on Wireguard but offers an even simpler configuration: install clients on your machines, log onto Tailscale with your credentials, done. The best part is that you don't even have to deal with setting up a NAT on your private network, enabling packet forwarding or firewall rules of any kind. If all you want is a working setup, take some time to read through their docs. The only downside I found is that updating the client often requires re-authenticating to tailscale, interrupting the service without warning. It is Ok for a smartphone VPN client but not really cool for 24x7 servers running without much supervision.</p>
<p>There are other advantages to running Tailscale vs running Wireguard like out-of-the-box support for mesh configuration. Check both out before you make up your mind on a given VPN solution.</p>
<p>Now let's assume you are running your own VPN between your VPS and your home server. Your next step will be to install a reverse proxy on your VPS to forward all requests made to a given port or host name to a specific address:port combination on your private network. Something like: incoming requests for myblog.mydomain (80 or 443) get forwarded to 10.10.10.2 (same ports), which is the address of your home server on your VPN. Internet users only see the VPS public address, your home server only listens to its VPN address and is not reachable from anywhere except other machines on the same VPN.</p>
<p>The beauty of that approach is that you can host several services from multiple domains on the same public IP address and let the reverse proxy do its job and forward to whatever home server you happen to run. You can even request server certificates from <a class="reference external" href="https://letsencrypt.org/">LetsEncrypt</a> and install them all on your reverse proxy, then run a straight non-TLS connection between your VPS and home server. That line is already protected by your VPN.</p>
<p>Several choices are available for reverse proxies. I tend to use <a class="reference external" href="https://nginx.org/en/">nginx</a> which is a real breeze to set up.</p>
<p>If you need to forward other ports or other protocols you can use a bare-bone solution like <a class="reference external" href="https://man.archlinux.org/man/socat.1.en">socat</a> to forward packets from the public IP on your VPS to an internal address on your VPN. Beware that socat will terminate as soon as the connection is broken, so best encapsulate it into a systemd service or <a class="reference external" href="https://stackoverflow.com/questions/9341254/reconnect-socat-connection-after-network-failure">simply a while loop in a shell script to automatically restart it when needed</a>.</p>
<p>The final result is quite nice: the only visible IP address is the one you rent with your Virtual Server, on your choice of DNS name and port. If it gets caught into a DDoS storm, your service will become unreachable but your home Internet stays fine. Many VPS providers offer DDoS protection tools which may be efficient enough to stop most such attacks.</p>
<p>Meanwhile, your home address is only connected through a VPN based on UDP packets, which makes it a lot harder to discover by stupid Internet-searching robots. Using a solution like Wireguard is unlikely to eat up lots of CPU/RAM on your home server: in my experience a Raspberry Pi 4 is largely enough to saturate a 250Mbit/s connection. If you are paranoid, you can run Virtual Machines or containers on your RPi4 to encapsulate your services and limit damages in case someone manages to take control of your VPS. Make sure you run automatic updates on that VPS Linux box because you won't log onto it every day.</p>
<p>If your home connection is slower than what your VPS can serve, you also probably want to rate-limit that VPN connection if you don't want to flood your home Internet connection the day your service becomes popular. Some good tutorials should help there.</p>
<p>I have no pointed any particular tutorial in that post as new good ones keep popping up every day and I am too lazy to do a good research that will become obsolete in a year.</p>
<p>Enjoy your public home services!</p>
btrfs for home usage2021-05-16T22:42:00+02:002021-05-16T22:42:00+02:00nicolas314tag:maz.one,2021-05-16:/btrfs-for-home-usage.html<div class="figure">
<img alt="Some of my currently unused 2.5-inch drives" src="/images/disks-1.jpg" />
<p class="caption">Some of my currently unused 2.5-inch drives</p>
</div>
<p>Over the years I have accumulated so many hard drives at home it is getting ridiculous. Last time I went through a decommissioning frenzy I must have dumped at least twenty of them to the tech recycle bin. Sizes ranged from several …</p><div class="figure">
<img alt="Some of my currently unused 2.5-inch drives" src="/images/disks-1.jpg" />
<p class="caption">Some of my currently unused 2.5-inch drives</p>
</div>
<p>Over the years I have accumulated so many hard drives at home it is getting ridiculous. Last time I went through a decommissioning frenzy I must have dumped at least twenty of them to the tech recycle bin. Sizes ranged from several hundred megabytes to several hundred gigabytes. The drives were too small to be of any use, too power-greedy, noisy, slow (looking at you old 4GB bigfoot), or just plain defective (why did I even keep them?) with that dreadful <em>tick-tick-tick</em> music you never want to hear from a live machine.</p>
<p>Before throwing disks away I take care of erasing their contents while they can still be reached on a USB adapter. No need for complicated methods, a simple zeroing does the job:</p>
<p><tt class="docutils literal">dd <span class="pre">if=/dev/zero</span> <span class="pre">of=/dev/sdX</span> bs=4M status=progress</tt></p>
<p>I will sometimes recycle the powerful magnets inside to stick pictures to my fridge, but unscrewing those little boxes can take some time.</p>
<p>Anyway: I was still left with a couple of perfectly functional terabyte-sized drives and decided to have some fun with filesystems because why not? I got a 4-bay USB3 enclosure for cheap on Amazon, installed the drives, and connected it to a Linux box. ZFS seemed like an interesting thing to learn but that box runs Debian: I would have needed to jump through hoops to install ZFS support and didn't feel so adventurous. I opted instead for the lesser-known btrfs, which has apparently been in the Linux kernel for over a decade now.</p>
<p><a class="reference external" href="https://en.wikipedia.org/wiki/Btrfs">https://en.wikipedia.org/wiki/Btrfs</a></p>
<p>Seen from wikipedia, btrfs comes with most of the ZFS goodness, and sometimes more: snapshots, live checksumming, subvolumes, perfect solution to experiment building a simple array with just two (identical) terabyte-sized drives. Long story short: after a couple of months or so this thing ate all the data I copied to it. I only copied test files so nothing lost, and I learned quite a few things in the process.</p>
<p>In more details:</p>
<p>Whenever a btrfs filesystem is mounted you immediately get a dozen or so processes running full time and eating away at your CPU. I suppose they do some housekeeping like computing and validating checksums in the background. Nothing wrong with that but I will object to having a constant CPU load due to a filesystem. It would have been great to be able to control when housekeeping takes place but it either does not exist or I overlooked it.</p>
<p>At some point the disks started failing, and I only noticed because the filesystem had been remounted as read-only. Sure, 'dmesg' gave me a long list of access/read errors all over the place, but I didn't get any particular warning as a user until it was impossible to write a new file to that directory. A better user interface could have been to run a cron job to check for errors and send me an email when failures are detected? I managed to get that working with weekly SMART reports but could not find or overlooked any way to do that in the btrfs docs.</p>
<p>Once the drives had failed I could still read about half of the files. The other half was gone. Best I could do was 'ls' in a directory and get a screenful of question marks all over the place, indicating that the file system was corrupted. I tried copying some of the readable files but a quick MD5 indicated that they were corrupted too. Tough! Turning the machine off and on again didn't help: the filesystem would go read-only within a couple of minutes.</p>
<p>Ok so those drives are old: maybe 7 or 8 years, but their SMART reports indicate absolutely no errors. I ran daily short and weekly long off-line tests all along and all lights always came up green. I have now re-formatted them to ext4 and they are happily humming in the background as I type this, checking for bad blocks. Wait and see if there are more errors with a different filesystem.</p>
<p>Now to be fair: this is just a data point, and not a very scientific one at that. I could certainly have spent more time in the documentation to find out about how to get regular reports and what to do when corruption is detected, or even run some post-mortem analysis tools to understand what went wrong. This is certainly something I would have done in a professional context, just not at home when all I am trying to do is get a feel for a fun filesystem.</p>
<p>Let's be clear: btrfs sounds like a lot of fun. The notion of subvolumes within a volume is great, think of it like partitions that have no quotas. This can be useful when you only want to backup a subset of the drive. btrfs also has the notion of send and receive (like ZFS), allowing to easily make backups on live systems without stopping anything. You can also live-detach a drive from a group or add a new drive while the filesystem is under use. I haven't tried any of that but I could imagine myself using that on production servers.</p>
<p>The main reason why I wanted a checksumming filesystem was to get warnings before bitrot starts eating through my archives. Unfortunately I didn't get any of that and I still don't know if the corruption I got was purely due to me messing up the btrfs settings (didn't do much of that) or defective drives. Time will tell, but the point is made: a btrfs filesystem can go wild without any warning.</p>
<p>My conclusion: this filesystem seems still too rough for your average casual user. Like all professional tools, it requires some investment if you want to understand what is going on under the hood and how to best use it to serve your needs. To be fair, I would expect to get bitten just as well by a ZFS install on the same hardware. Life is short, I guess ext4 will have to do the job for now.</p>
My very own little Internet2021-03-24T23:52:00+01:002021-03-24T23:52:00+01:00nicolas314tag:maz.one,2021-03-24:/my-very-own-little-internet.html<p>I watched this Netflix film about the flat-Earthers and found it quite enjoyable. Having been educated in the sciences, I marveled at that sorry lot trying to prove the Earth flat and invariably ending up approximating its radius. Made me wonder: how do <strong>I</strong> know for a fact that we …</p><p>I watched this Netflix film about the flat-Earthers and found it quite enjoyable. Having been educated in the sciences, I marveled at that sorry lot trying to prove the Earth flat and invariably ending up approximating its radius. Made me wonder: how do <strong>I</strong> know for a fact that we live on an imperfect sphere? Few things come to mind: take a late plane, fly along the terminator, the limit between day and night, and watch the curvature from 10 kms above. Can't make that up. And if the Earth was flat, how would you explain time zones, seasons, or why the Moon, the Sun, the planets are all spherical? Couldn't very well explain gravity either.</p>
<p>Which brought me to: does this flat-Earth club really exist? The next thing I did ruined my YouTube experience for life. I simply searched for "Flat Earth society" through the YouTube app on my tablet. Of course I got to see the real videos they put online, which were a bit scary but still a lot of fun to watch. Little did I know that YouTube had just classified and locked me into the little but growing box of conspirationist nuts.</p>
<p>For the next few weeks, when I called up YouTube on that tablet I would be fed with endless content about conspiration theories and NOTHING ELSE. I cleaned the tablet, re-installed Android, said prayers to the Android gods, but nothing seemed to change its point of view about my video tastes.</p>
<p>Quite annoying. I also noticed the same thing had spread to my other devices. Anything I would want to watch on YouTube was invariably followed by a very serious documentary about mysteries of the Great Pyramids or why Neil Armstrong never walked on the Moon. There is so much bullshit content on YouTube that I could not click it away, and all my search results were videos made from conspirationists about whatever seems to be the popular story today. That got me completely off YouTube for quite a while, which is not such a bad thing.</p>
<p>Things are going better now. For some reason, YouTube is convinced I am a British (wrong) black (wrong) woman (wrong) in her sixties (wrong) living as an expat (kind of) in France (true, half of the time). It keeps offering me good deals for French lessons (non merci) and tax advice for expats (nope). See just below for a taste of the kind of ads I see whenever I open YouTube:</p>
<img alt="Youtube ad" src="/images/expat.jpg" />
<p>If you think this stops at YouTube, think again. I stopped using Google Search for the very same reason: the results are tailored to my "tastes" whether I want it or not. Calling up the same search terms on various computers brings different results. French Google, UK Google, and German Google have very different ideas about what you need to know than US Google. And before you ask: yes, I make sure I am never logged into a Google account when using Search, precisely to avoid being fed a narrow-minded view of the world computed by some obscure algorithm trained on LSD.</p>
<p>I later watched <em>the Social Dilemma</em>, a film that explains quite a lot. Social networks need to maximize the time you spend on them so you can see more ads, so they train their algorithms to only feed you what they think you want to see or read. And that, folks, is the end of the era of information.</p>
<p>In technical terms, information entropy is defined as measuring the amount of surprise you get from a new message. No surprise, no entropy. When Google keeps serving me the same piece of information ad nauseam I am not learning anything new, it just tells me that I am not using the right source. There used to be a time when search engines were only about showing you what they had in store related to your search terms. Now they lock everyone into their own tailor-made tiny bubbles and make them believe the rest of the world is in perfect harmony with them. Nothing could be more stupid. <em>The Social Dilemma</em> explains how this is ripping apart the fabric of democracy, but I would argue that this kind of behaviour is also destroying the value of those very search methods.</p>
<p>Let me give you another example: I recently met a guy who ended all arguments by pulling out his iPhone and looking up the very words he just said. He would then confidently brandish his phone at me: "see? I was right, this is all over the Internet!". Man, if you had typed just the opposite argument you might have found just as many hits. I am waiting for the moment when two people facing each other pull out their phones, type the same search request and get opposite results because they are tailored to their own tastes. That day, those two might realize that Google is just a big advertising company and not the source of all human knowledge. <em>Do no evil</em> doesn't mean you can't be daft and consider your users as cattle.</p>
<p>DuckDuckGo is a better search engine, doesn't track you and doesn't try to show you only what it thinks you want to see. Problem solved for search engines. Now how are we going to solve that issue with social media? Is it even possible?</p>
<p>The year is 2021 and people can now train neural networks to perform amazing things. The only trouble is: we have no idea how or why they act as they do. Teach an AI to take decisions based on various inputs and you will certainly get an answer. Now why was that particular answer chosen? All you have is a bunch of floating-point values without any meaning. Good luck attaching a <em>why</em> to those values.</p>
<p>Here is a real-life example: a few years back I attended a talk given by two researchers who wanted to train an AI to identify suspicious traffic on a company's network. They fed it a day of normal traffic, then a day of virus-infected traffic, and so on until the network was considered ready for testing. The resulting algorithm was surprisingly fast and the outcome was rather simple: odd days were classified as normal, even days as suspicious, just because the initial data set alternated every day between normal and suspicious traffic.</p>
<p>That's an easy rookie mistake to spot, but data poisoning can be more insidious and almost completely impossible to detect. If we want to stay away from Terry Gilliam's <em>Brazil</em> movie where lives are ruined by a single Buttle/Tuttle character change, we need to take things into our own hands and poison the well ourselves.</p>
<p>My take on it: many stores in Paris now ask you for your postal code. I always answer: 46800, which corresponds to several villages including the lovely town of Montcuq. It is always a pleasure to see the face of the poor cashier browsing through the village list and asking me: "where do you live Sir?", and answer with perfect composure: "mon cul".</p>
<p>Other good candidates are: Brest (29200), Monteton (47120), Condom (32100), Bitche (57230), Trécon (51130), or Craponne (69290).</p>
<p>Every year on New Year's eve, I take pleasure in seeing my mailbox flooded with happy birthday emails from all over the globe because my default birth date for all sites is 01/01/1970, i.e. the Unix epoch. My email addresses are all from the example.com domain. When a US web site asks for a postal address I provide the first pizzeria in New York that pops up on Maps.</p>
<p>Time to react, folks! Let's pollute those data wells! Let's play cat-and-mouse with the advertisers and make their life miserable until they find a business model that does not ultimately destroy us.</p>
Fun with ESP322021-01-27T23:03:00+01:002021-01-27T23:03:00+01:00nicolas314tag:maz.one,2021-01-27:/fun-with-esp32.html<div class="figure">
<img alt="ESP32" src="/images/esp32.jpg" />
</div>
<p><a class="reference external" href="https://www.espressif.com/">Espressif</a> is this little fabless Chinese company who are behind the incredibly successful <a class="reference external" href="https://en.wikipedia.org/wiki/ESP8266">ESP8266</a> chip, followed in 2016 by a more powerful successor: the <a class="reference external" href="https://en.wikipedia.org/wiki/ESP32">ESP32</a>. This microcontroller allegedly uses a lot more power than equivalent Arm MCUs (though I haven't done any research on that topic), but what really makes …</p><div class="figure">
<img alt="ESP32" src="/images/esp32.jpg" />
</div>
<p><a class="reference external" href="https://www.espressif.com/">Espressif</a> is this little fabless Chinese company who are behind the incredibly successful <a class="reference external" href="https://en.wikipedia.org/wiki/ESP8266">ESP8266</a> chip, followed in 2016 by a more powerful successor: the <a class="reference external" href="https://en.wikipedia.org/wiki/ESP32">ESP32</a>. This microcontroller allegedly uses a lot more power than equivalent Arm MCUs (though I haven't done any research on that topic), but what really makes a difference with similar Arm-based microcontrollers is the price point. Boards based on the older ESP8266 chip can be bought for a bit less than 2€ (+shipping) from AliExpress if you can stomach the wait (1-2 months) or for hardly more from Amazon if you commit to a 5-pack, with next-day or even same-day delivery.</p>
<p>ESP32 refers to the microcontroller chip that can be found on a whole variety of products. Some creative Chinese companies have built countless variations around that theme, which can make it a bit difficult to choose the right board if you don't know what to look for.</p>
<div class="section" id="some-specs">
<h2>Some Specs</h2>
<p>The chip (officially called Xtensa) is actually a 32-bit dual-core running at 240MHz, which is thirty times faster than <a class="reference external" href="https://en.wikipedia.org/wiki/PC1512">my first IBM-compatible</a>. It has 540kB of embedded RAM and comes with no internal flash. No need to worry about flash storage though: all the boards you can find add their own on-board flash ranging from 4MB (standard) to 16MB (rare).</p>
<p>Some boards also add external RAM connected through an SPI bus: most often a 4MB chip, sometimes up to 8MB, referred to as SPIRAM. Careful though: the processor can only work with 4MB chunks at a time and you will need to care about bank switching if you want to use the upper 4MB of an 8MB RAM chip.</p>
<p>You can have up to 44 programmable GPIOs, though most often some are reserved for those extra flash, RAM chips, user LEDs, or even a micro-SD card interface. Some GPIOs are reserved for analog signals, most are meant for digital usage. Some pins are input-only. Good thing is: GPIOs are not dedicated to a single usage like a lot of Arm microcontroller boards, so you can decide to put your I2C or UART on any pin you like, even assigning them dynamically at run-time.</p>
<p>In addition to the dual-core, the MPU also has a low-energy mode described as ULP (Ultra-Low Power), which takes over most of the time when the processor is not doing anything interesting. This manages to keep power usage low, making ESP32 good candidates for battery-powered projects. The ULP quickly wakes up from deep sleep upon interruptions so you are guaranteed to never miss a beat from a sensor. This may explain why those chips are so popular for real-time systems.</p>
<p>Full specs can be found on <a class="reference external" href="http://esp32.net/">http://esp32.net/</a></p>
<div class="section" id="things-you-need-to-know">
<h3>Things you need to know</h3>
<p>Most ESP32 boards come with an integrated micro-USB or USB type C connector that can be used for both power and serial console. If you connect it to a Linux PC you can immediately talk over the wire using tools like screen, minicom, or picocom, e.g.</p>
<pre class="code literal-block">
picocom -b 115200 /dev/ttyUSB0
</pre>
<p>Putty is recommended on Windows, screen picocom and minicom also work on Mac.</p>
<p>Some boards don't have any USB connector. You can find really cheap USB-to-serial adapters on Amazon for a couple more quids, which you have to connect using wires to the right pins.</p>
</div>
<div class="section" id="boards">
<h3>Boards</h3>
</div>
</div>
<div class="section" id="wemos-d1-mini">
<h2>Wemos D1 Mini</h2>
<p>Starting from the cheapest available board: the <a class="reference external" href="https://www.teachmemicro.com/getting-started-wemos-d1-mini">Wemos D1 Mini</a>.</p>
<div class="figure">
<img alt="D1 mini" src="/images/d1mini.png" />
</div>
<p>This one is really cheap, I got a couple from an electronics store in Paris for 2€/piece. They don't provide so many GPIOs so they are better suited for projects that embed a couple of sensors. The <a class="reference external" href="https://components101.com/dht11-temperature-sensor">DHT11</a> and <a class="reference external" href="https://components101.com/sensors/dht22-pinout-specs-datasheet">DHT22</a> are cheap (2 to 4€) popular sensors for temperature and humidity, though I would recommend the DHT22 which has sub-degree precision where the DHT11 only delivers integer values.</p>
</div>
<div class="section" id="esp32-wroom32">
<h2>ESP32 WROOM32</h2>
<p>For a handful more cents you can get what is probably the most popular ESP32 board.</p>
<p>It has a red programmable LED, which can be useful to signal what is going on, and more GPIOs than the D1 mini. No extra RAM but has 4MB of flash storage.</p>
<div class="figure">
<img alt="Wroom 32" src="/images/wroom32.png" />
</div>
</div>
<div class="section" id="ai-thinker-esp32-cam">
<h2>AI Thinker ESP32 Cam</h2>
<p>For five to ten euros, this board embeds a tiny camera capable of delivering up to 1600x1200 pixels at a high frame-rate without a sweat. It embeds 8MB of RAM (so two 4 MB banks) and 4MB of flash.</p>
<div class="figure">
<img alt="AI Thinker" src="/images/aithinker.png" />
</div>
<p>You can download and run pre-packaged firmware that allows you to put it immediately online on your home network and steer it from your web browser.</p>
<p>That thing also has a very bright LED that can be used as flash, but it is really bright and hurts to look at. It does not have any USB port so you need your own USB-to-serial adapter to program it, and some solution to power it up through the pins.</p>
</div>
<div class="section" id="ttgo-esp32-t8">
<h2>TTGO ESP32 T8</h2>
<p>The default version comes with 4MB flash and 4MB RAM, a micro-USB connector and an SD card reader.</p>
<div class="figure">
<img alt="TTGO T8" src="/images/ttgo-t8.png" />
</div>
<p>It is often sold together with a small LCD screen that makes it attractive for creating your own GameBoy emulators or equivalent. 10-20€ from Amazon, or less than 10€ from AliExpress.</p>
<div class="figure">
<img alt="TTGO display" src="/images/ttgo-display.png" />
</div>
<p>Of course the screen controls eat a few GPIOs but the screen quality is excellent and perfectly usable to display small animations or even videos.</p>
</div>
<div class="section" id="m5stack-family">
<h2>M5Stack Family</h2>
<p>The M5Stack family of devices offer a completely different experience: devices are built into small 5cmx5cm boxes with all GPIOs exposed on the sides. The default core piece comes with an attached LCD, a buzzer, three buttons, a USB-C connector, an SD card reader, but no extra RAM. You can add more pieces and combine (stack) them like Legos to build more complex projects.</p>
<div class="figure">
<img alt="M5 stack" src="/images/m5stack.png" />
</div>
<p>I got a couple of those basic cores, they are fun to play with but not suited for real development. If you install the Arduino IDE, they are completely supported and come with a whole lot of games and utilities like a WiFi scanner, a weather station, and other completely useless and fun programs that can be dynamically loaded from the SD card.</p>
<p>Tough point: their price. Count 25€/piece for basic stuff, up to 50€ for more advanced sensors like GPS or air quality sensors. For that kind of price I would get raspberry zeros and forget about the pain of programming and debugging microcontrollers.</p>
<div class="section" id="programming">
<h3>Programming</h3>
<p>There are many options for programming those little critters, with the most popular being:</p>
<ul class="simple">
<li>The Espressif ESP32 SDK</li>
<li>The Arduino IDE</li>
<li>MicroPython</li>
<li>Microsoft Visual Studio</li>
</ul>
</div>
</div>
<div class="section" id="the-espressif-esp32-sdk-esp-idf">
<h2>The Espressif ESP32 SDK: ESP-IDF</h2>
<p>The SDK is directly available from github: <a class="reference external" href="https://github.com/espressif/esp-idf">https://github.com/espressif/esp-idf</a></p>
<p>You can use C or C++ and it comes with a lot of useful examples to help you get started with WiFi, BlueTooth, GPIO management and protocols of all kinds. It is supposed to work on Linux, Mac, and Windows though I only tried it on Linux.</p>
<p>I faced some trouble with that environment. There are currently two major versions under maintenance: V3 and V4.2. Version 3 has several notorious bugs, e.g. preventing usage of both BlueTooth and WiFi simultaneously. The same antenna is used for both, you need to multiplex access to it, sometimes with unfortunate results. I have never managed to get simultaneous BT+WiFi connections for more than 30 seconds.</p>
<p>Version 4.2 is far more advanced and apparently corrects a few of those issues. If you are going for a pure C-based environment, I would suggest starting from examples, copy/paste what you need until your app is complete.</p>
<p>Most of the trouble comes from the fact that Espressif keeps updating the SDK about twice a year without any care for backwards compatibility. If your project works with SDK version X.Y you are stuck with that SDK. If you want to upgrade to another version you will have to port your own code because they changed a whole lot of APIs. They can also be really creative with the build system: a mixture of Makefiles, CMakefiles, Kconfig, topped with Python scripts that require a zillion dependencies to run. Prepare for an install frenzy of Python packages for the first couple of hours. And of course those Python packages change (a lot) from one version of the SDK to another. Ouch.</p>
<p>The build system is rather brittle and will explode in your face at regular intervals, leaving you with no choice but spending a good part of your evening learning about the intricacies of CMake and Kconfig.</p>
<p>The SDK uses FreeRTOS under the hood, but if you stick to the documented APIs you don’t even have to care about that. I never had any trouble with FreeRTOS.</p>
<p>User-facing APIs are well documented and a joy to read:</p>
<p><a class="reference external" href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/">https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/</a></p>
<p>They have English-speaking proofreaders and it shows, as opposed to documentation found for a lot of ESP32 boards that sometimes look like hard-to-understand translations from Chinese..</p>
</div>
<div class="section" id="the-arduino-ide">
<h2>The Arduino IDE</h2>
<p>This is probably the recommended way if you want to stay close to the metal without having to deal with the tool dependency and build system nightmare. Download and install the standard Arduino IDE, add an Espressif URL somewhere in your config, click to pull out all the toolchain, and you are done. You can find examples directly from the File/Examples menu, compile them and run them on a USB-connected board.</p>
<p>Using the Arduino IDE produces code that takes a bit more space in RAM and flash on devices because of their fancy C++ layer. It does not hurt but I just don’t like the IDE itself or the fact that it hides a lot of bare-metal things from me. It is Ok for playing and testing ideas, doing throwaway stuff, but I would not trust it for real projects that need to be maintained over long periods of time, with version control or complex library management. On the plus side you have access to pretty much all ESP32 libraries in existence, including some NES emulators (if your board has a screen) or support for exotic sensors without having to browse a million websites before you find that obscure driver. That makes it an ideal environment for trying out new hardware, or if your projects are more about hardware than software.</p>
</div>
<div class="section" id="micropython">
<h2>MicroPython</h2>
<p>Talking about staying away from the metal: Python is definitely a viable language on this platform with MicroPython. This version is a complete rewrite of the Python3 interpreter for microcontrollers. It also comes with batteries included (tons of fancy packages) but also has some of its own quirks.</p>
<p>You can theoretically use MicroPython on a dead-simple ESP32 with no extra RAM, but the interpreter + FreeRTOS + libraries only leave you less than 100kB of RAM to play with, which is excessively small. I tried to use it on a core M5Stack to display images on the LCD screen, only to find out that a lot of JPG images do not have enough space to decompress in RAM. You can work around that by striping the decoding but it requires diving deep into the JPEG library, which happens below the Python layer. Back to C!</p>
<p>On TTGO devices you have more RAM (4MB) and it becomes completely viable and usable. Once you flash the device with MicroPython, you can upload your Python source files using a simple command-line tool (ampy) or even read them directly from an SD card if present. Put your code in main.py and your app will automatically start after boot.</p>
<p>I found two version of MicroPython to play with: LoBoris MicroPython and mainline MicroPython.</p>
<p>LoBoris MicroPython binaries can be downloaded for ESP32:</p>
<p><a class="reference external" href="https://github.com/loboris/MicroPython_ESP32_psRAM_LoBo">https://github.com/loboris/MicroPython_ESP32_psRAM_LoBo</a></p>
<p>It works fine everywhere I tried it, but the project seems unmaintained today and uses an old version of MicroPython. I could not manage to compile it from scratch, which is a non-starter since I would like to add my own C-based modules. There might be a way but I found mainline MicroPython easier to tinker with.</p>
<p>The official MicroPython works on all kinds of microcontrollers:</p>
<p><a class="reference external" href="http://micropython.org/">http://micropython.org/</a></p>
<p>MicroPython makes it ridiculously simple to program sensors. Here is an example for a DHT11 temperature and humidity sensor connected to GPIO 2:</p>
<pre class="code literal-block">
import dht
while True:
sensor = dht.DHT11(Pin(2))
sensor.measure()
print('Temperature = %d' % sensor.temperature())
print('Humidity = %d' % sensor.humidity())
time.sleep(3)
</pre>
<p>That’s it. No main(), no build system, no initialization, and you can dynamically change the PIN you attached the sensor to. Since Python is a high-level language you can do a lot in very few lines of code so RAM usage for your code itself should not be an issue.</p>
<p>If you still need to stay close to the metal for any reason, you can also build your own C modules and make them callable from Python. There is a lot of boilerplate involved but you can find good online tutorials about how to do this.</p>
</div>
<div class="section" id="microsoft-visual-studio">
<h2>Microsoft Visual Studio</h2>
<p>I had never used that IDE before but it seems to work fine. I installed it under Linux in a matter of minutes and then got completely overwhelmed by the huge number of menus, options, and blinking lights. I tried to follow a tutorial to set up the ESP32 environment but miserably failed, could not manage to compile anything, and gave up after a couple of hours. Probably worth another try cause everybody else says it is great.</p>
<p><a class="reference external" href="https://platformio.org/">PlatformIO</a> offers additional tools for doing embedded software with Visual Studio. Same result: the number of options, menu, and popups made it too overwhelming for me to achieve anything. After an hour of looking through tutorials and examples I still had not managed to compile anything. Those fancy IDEs are just not my thing I suppose.</p>
<div class="section" id="honorable-mentions">
<h3>Honorable Mentions</h3>
</div>
</div>
<div class="section" id="lua-rtos">
<h2>Lua-RTOS</h2>
<p>Honorable mention for this other project based on the Lua language: <a class="reference external" href="https://github.com/whitecatboard/Lua-RTOS-ESP32">Lua-RTOS</a></p>
<p>Lua is a tiny programming language meant to be easily learned and embedded into other programs. The interpreter compiles to a 100-200KB binary on Linux, less if you remove some unneeded libraries. Going through the tutorial makes you a Lua professional in a couple of evenings. The language is about as high-level as Python with less bells and whistles, and gets the job done. Lua itself is written in C and integrates very easily with C libraries without too much boilerplate. Lua is based on a virtual machine that is a joy to look at. The code is clean, small, and very understandable.</p>
<p>I wish I could use Lua-RTOS out of the box but unfortunately I could not get the pre-compiled binary they provide to support an SD card reader on a TTGO ESP32. You need to recompile it from scratch based on a particular tag of the Espressif V3 SDK, which brings you back to fun with the Espressif SDK V3 build system. I did not manage to compile a working version with SD card support because I could not find the right options to remove unwanted packages to save some space, and the resulting binary did not fit. In the end I took the hard way and integrated a straight Lua interpreter directly into the system/console example provided with the Espressif SDK, and managed to get a Lua prompt working over serial. It took me a whole evening to do that and the result was not too impressive. If I wanted to add support for easy WiFi connections, network information, or GPIO support, it would probably take a lot more time. Maybe another project one day.</p>
</div>
<div class="section" id="tasmota32">
<h2>Tasmota32</h2>
<p><a class="reference external" href="https://tasmota.github.io/docs/">Tasmota</a> is an embedded OS initially written for ESP8266. More recently, the OS was ported to ESP32 as Tasmota32. It is designed to be a firmware alternative for pre-packaged connected objects such as connected lamps and power plugs. You can also run it directly on any ESP32 board for great results.</p>
<p>Tasmota is meant to operate completely over the air: when you first boot the device it broadcasts its own WiFi SSID without password. Connect a phone to that network and access a configuration page where you can fill in the details for your home WiFi, then reboot the device and that’s it. Your device can now be controlled from an embedded tiny web server that allows you to switch it on and off. It has native MQTT integration so you can also control it from things like HomeAssistant on your home network, or even Alexa or Google Agent if that’s what you want.</p>
<p>I tried Tasmota32 with an ESP32-Camera and it works remarkably well. Device settings can be controlled directly from a browser, and images (or streams) can be captured by simply getting a fixed URL, making it trivial to integrate with home automation systems.</p>
<div class="section" id="esp32-madness">
<h3>ESP32 Madness</h3>
<p>Enthusiasts have taken the ESP32 to unmatched levels.</p>
<p><a class="reference external" href="http://www.lilygo.cn">LilyGo</a>, a Chinese company, offers some really cool ESP32-based hardware like a programmable watch, a tiny serial terminal, or an E-ink WiFi screen.</p>
<p>Yes you can now wait for OTA updates on your watch too!</p>
<img alt="ESP32 wearable" src="/images/esp32-wearable.png" style="width: 325px; height: 323px;" />
<div class="figure">
<img alt="VT100 clone" src="/images/vt100clone.png" />
</div>
<p>Cute VT-100 clone that fits the palm of your hand.</p>
<div class="figure">
<img alt="ESP32 with e-ink screen" src="/images/esp32-eink.png" />
</div>
<p>E-ink is good for long-term display without using any battery.</p>
<p>For really amazing stuff you can follow Bitluni on YouTube. That guy <a class="reference external" href="https://www.youtube.com/watch?v=T_n8PtMMLiQ">uses the analog signals to generate pictures on an oscilloscope in XY mode</a>, or to <a class="reference external" href="https://www.youtube.com/watch?v=G70CZLPjsXU">directly generate a perfectly valid VGA signal and sound.</a></p>
<p>You can find instructions for VGA mode on <a class="reference external" href="https://www.instructables.com/ESP32-Basic-PC-With-VGA-Output/">https://www.instructables.com/ESP32-Basic-PC-With-VGA-Output/</a></p>
<p>If you don’t want to solder it yourself, you can find ready-to-use VGA+PS2 adapters from LilyGo.</p>
<p>Old 8-bit games run fine too if you have a screen and a gaming controls:</p>
<figure><div class="docutils container">
<a class="reference external" href="https://hackaday.com/2020/06/09/run-your-favorite-8-bit-games-on-an-esp32/">https://hackaday.com/2020/06/09/run-your-favorite-8-bit-games-on-an-esp32/</a></div>
</figure></div>
<div class="section" id="worth-noting">
<h3>Worth noting</h3>
<p>The next generation of ESP32 is the <a class="reference external" href="https://www.espressif.com/en/news/ESP32_C3">ESP32-C3</a>, this time based on a RISC-V design. Espressif is promising more hardware oomph and an SDK based on RISC-V. The move to RISC-V means more reliance on standard toolchains to ease the pain of maintaining their own SDK. Don’t hold your breath though: the chip has just been announced in December 2020 and boards have not started to appear on the market yet.</p>
<p>Given the current price of ESP32 boards you have nothing to lose anyway by trying them yourself and get a taste of the various development environments.</p>
</div>
</div>
Home Automation2020-01-08T21:46:00+01:002020-01-08T21:46:00+01:00nicolas314tag:maz.one,2020-01-08:/home-automation.html<img alt="Connected plug" src="/images/plug.jpg" style="width: 371px; height: 341px;" />
<p>I was a bit late to the party, but I finally got into home automation and bought a couple of connected power plugs. Price and curiosity decided for me: they can often be found online for really cheap and they are a great excuse to spend a week-end doing nothing …</p><img alt="Connected plug" src="/images/plug.jpg" style="width: 371px; height: 341px;" />
<p>I was a bit late to the party, but I finally got into home automation and bought a couple of connected power plugs. Price and curiosity decided for me: they can often be found online for really cheap and they are a great excuse to spend a week-end doing nothing else but play with new gadgets.</p>
<p>The plugs I got are from a noname Chinese company. The packaging is raw cardboard with no indications, and the manual is limited to a thin and small sheet of paper with a QR code that sends you to the app store to download an app called SmartLife. So be it.</p>
<p>Start the app and get faced with a login screen that forces you to create an account and register your product. Pain in the butt. Once logged in, you need to identify the kind of device you want to register. That's a bit surprising. You would expect connected devices to be able to exchange their pedigrees over the net instead of me having to shuffle through thousands of possible products and probably getting it wrong. Alright, moving on.</p>
<p>Next step is device registration: put the app in pairing mode, press the only button on the plug long enough, and magic: your connected power plug now appears inside the app. Great!</p>
<p>Let's spice it up a bit: the plugs are advertised as being Google-compatible, so start the Google home assistant and try the additional linking process. The Google menu is scary: there are literally hundreds of IoT device vendors out there, and I have no idea which one I should be looking for. No brand name on the box or on the plugs, and the product name displayed on Amazon does not appear anywhere in the endless list proposed by the Google app. Hmm... Let's try with SmartLife? Yes: Google knows about SmartLife and asks for full control over the account I just created above in the app. Why not, who cares? Be my guest.</p>
<p>Once this was done, the two plugs appeared magically under Google Home and I could finally say "Hey Google, turn the lamps on!" and get enlightened. Phew!</p>
<p>That's fun but I have no intention of leaving a device that permanently records every sound in my house, so voice command is pretty much out of the picture. So without voice command, let me summarize how to switch the lights on in my living room in 8 easy steps:</p>
<ul class="simple">
<li>Since the app is registered with a SmartLife and Google account, I only installed it on one device. Which means I am the only one who has the power to switch lights on and off. Which also means that I am the only one who has to rummage through the house to find where I left my bloody smartphone again.</li>
<li>Unlock phone with fingerprint. Try again. Nope, try again. Give up and type the password. Avoid the temptation to read the bazillion notifications.</li>
<li>Find the app. How's it called again? Go through the endless list of apps, cursing yourself for not putting in on the home screen with the other 255 must-have apps.</li>
<li>Open the app, but since you haven't used it for a while you need to login again. Of course you forgot the password, so click "Forgot password" and get a link by email to reset it within 5 minutes. Hopefully. Now is the right time to sort out those bazillion notifications.</li>
<li>Now you got in, making a mental note to write that password down somewhere on a piece of paper and also remember where you left that piece of paper. Select the <em>Devices</em> tab, go to the lamp you want to switch on, and click.</li>
<li>Nothing happens, so click again. And again, several times. Press even harder on the phone screen, you never know.</li>
<li>Now all your clicking finally gets through and the lamp starts switching on and off like a Christmas tree, ending up switched off, of course.</li>
<li>Click again, then wait. Tadaaa, the light is on!</li>
</ul>
<p>Easy, right?</p>
<p>The next day I looked into advanced options and found a solution: use a timer. Instead of doing it myself, automate it, of course! Program the lamp to switch on when it's getting dark, and switch off at midnight. Problem solved.</p>
<p>Or maybe not.</p>
<p>Let's assume I am sitting in my living room, the lamp is 3 feet away, but just like a real lazy bastard I reach for my phone, open the app, and press the button. What happens next?</p>
<p>My phone connects to the SmartLife server. Could be in China, maybe in the USA, who knows?</p>
<p>I get authenticated. My order gets through, and a flag is raised somewhere on somebody else's servers, probably in the US on Amazon.</p>
<p>The plug in my living-room sends beacon packets to its home server every second, asking if it needs to do something. The raised flag finally gets to the plug, which acts and switches the light on.</p>
<p>How many things could go wrong?</p>
<p>My phone could stop working. Empty battery, bug in the app, bug in the OS, or it could just be the perfect moment to run a firware update that freezes the device for 20 minutes.</p>
<p>My phone may not be able to connect to the cloud. This requires an overseas link, which is of course not guaranteed to work. Same problem with the plug requesting commands every few seconds. Servers crash, databases get erased, latency can increase without warning. Seems stupid that a server crash in China would prevent me from turning the lamps on in my living-room. I do not want to subscribe to Amazon's server health status to figure out whether I will be in the dark tonight at home.</p>
<p>My home network is nowhere near industrial availability. Between the internet modem and the plugs are several switches, a router, and a WiFi access point that sometimes decides to take a few hours off for relaxation.</p>
<p>Depending on a bunch of companies to keep my service alive does not sound very future-proof. What happens when some of those companies go out of business? What is the plan then? Will we throw away all the devices that depend on it for a living? I was looking for a network-controlled device, not for a life-time service contract with a third-party company whose name didn't even appear on the box.</p>
<p>In terms of energy consumption this is not environmentally sound. The plugs keep sending small data packets back home, which is not much in terms of bandwidth but forces every device on the way to grind packets for no reason. An overseas link necessarily involves 15-30 hops one way, that's a lot of machines working to <em>not</em> light up my living-room.</p>
<p>I also don't feel too good about needing to create an account to protect the plugs from outside attacks. This is just opening a welcome gate for hackers to get into my network, and the price for closing that gate is to forego any usage of the plugs. Not good.</p>
<p>Fortunately there are solutions. Most connected plugs today are based on the ESP8266, a very popular chip in the maker crowd. Just google a bit and you find methods to erase and replace the firmware for those plugs with something that does not need to call home every second.</p>
<p>The project I used is tuya convert:</p>
<p><a class="reference external" href="https://github.com/ct-Open-Source/tuya-convert">https://github.com/ct-Open-Source/tuya-convert</a></p>
<p>The procedure is rather simple. I used an Ubuntu-based laptop and got everything sorted in just 15 minutes. No need to open the box or solder anything, it all happens over the air. I re-flashed the firmware with Tasmota, an alternative ESP8266 piece of firmware, done. The two plugs in my house are now mini web servers that can be automated using MQTT or simply controlled from a web browser. Most importantly: they do not require an Internet connection to work and they do not flood the local network with broadcast messages every second.</p>
<p><a class="reference external" href="https://github.com/arendst/Tasmota">https://github.com/arendst/Tasmota</a></p>
<p>I lost only one thing: the ability to switch the lights from outside my house. Can't say I will miss much.</p>
<p>This got me thinking: seems most IoT connected devices today are designed to be connected to the cloud. It is extremely hard to find something that could just be controlled from your local network without need to ping back to China or the US or who-knows-where. Why? I understand these companies are hungry for user data, but what's in it for me? Requiring an active Internet connection should not be a pre-requisite for switching lamps on!</p>
<p>One small note about security: those devices are advertised as being top-notch secure and based on military-grade security. And yet you can simply erase and re-flash their firmware over the air without much effort. Simple but deadly mistake: devices do not validate server-side certificates. It is rather trivial to start your own web server with a self-signed certificate and point the device to it using a well-configured local DNS server. This is great because it means I could get rid of the initial firmware, but it also means anyone in the same room could do the same and poison the firmware. Who knows how many other security issues these devices suffer from?</p>
<p>I can easily imagine that instead of relying on distant machines located in data centers, we could host the main control point ourselves in our houses. Modern houses already have Ethernet cabling running through all rooms, with some space in a closet to connect a router and a switch. It would not be hard to install an additional server there, something Raspberry-pi-sized, and use it to control every piece of connected hardware in the house.</p>
<p>You guessed it: there are already several projects on that topic. The most promising I found seems to be Home Assistant:</p>
<p><a class="reference external" href="https://www.home-assistant.io/">https://www.home-assistant.io/</a></p>
<p>I have not looked too much into it yet but will do soon, and maybe write it up here. Next evolution would be to be able to run voice recognition on that same local machine so we don't have to be spied upon by Google and Amazon. I am really looking forward to the next company offering a <em>Home OS</em>. Right now these home automation gadgets are just a jungle of services quickly strung together with no future.</p>
<p>Today's lesson: don't buy connected devices if you cannot flash them with alternative firmware.</p>
Japan Kaleidoscope2018-09-01T23:31:00+02:002018-09-01T23:31:00+02:00nicolas314tag:maz.one,2018-09-01:/japan-kaleidoscope.html<img alt="Akihabara" src="/images/japan-cover.jpg" style="width: 80%;" />
<p>I have just spent three weeks in Japan. What can you decently bring back
from such a short trip? I have the feeling I have only touched this
extremely rich and multi-millenial culture. A kaleidoscope of colours,
smells, sounds, and faces come to mind when I try to look back …</p><img alt="Akihabara" src="/images/japan-cover.jpg" style="width: 80%;" />
<p>I have just spent three weeks in Japan. What can you decently bring back
from such a short trip? I have the feeling I have only touched this
extremely rich and multi-millenial culture. A kaleidoscope of colours,
smells, sounds, and faces come to mind when I try to look back on this
small adventure. As usual, when I come back from Asia I discover that I
have taken home a little more than I bargained for. Some things are
changed, some things are broken inside me, only to be re-built from their
ashes.</p>
<p>During my first visit to Asia twenty five years ago, I got hit in the face
with fairly obvious cultural shocks. Things like counting on your fingers:
I wanted to tell the hotel staff we were three guests and raised my thumb,
index, and middle finger to indicate: THREE. The guy repeated with his
hand, looking surprised. He said something in Chinese (this was Taipei), I
said something in English, and we both understood we would be left with
gestures for the rest of the conversation. I waved again with my fingers:
THREE, so he repeated it, looking surprised. We were three guys standing in
the lobby so I didn't get it. I showed the two others: ONE, TWO, and
myself: THREE. The guy opened his whole hand, showing all fingers, asking
something. No we are not five, we are three. It took us a little while to
proceed, because what I had just gestured means EIGHT. The hotel clerk was
asking me where the other five were. In order to show three, you raise the
three middle fingers. Ah, right. I later learned how to count in Chinese,
an easy task: with just thirteen words you can count up to 999,999.</p>
<p>There were more embarrassing moments, like the first time I sat on a Korean
toilet and could not figure out how it worked since I am quite illiterate
in Korean. You may not want to hit random buttons and wait to see what
happens, unless you want to come out of the loo covered in smelly water
just before an important customer meeting.</p>
<p>Japan has those same magic seats that clean your ass like you are a baby.
Once you figured out the controls, you wonder why those are not generalized
in the Western world already. There is no better feeling than walking
around with a clean butt, especially in those hot summers with 40 degrees
and 100% humidity.</p>
<p>My major cultural shock during that last trip was food. If you have never
tasted food in Japan, you have never eaten decent food in your life. Ever.
The closest you can get to that in terms of taste, freshness, and diversity
could be Italy, on a very different spectrum. Japanese food does not just
build on fish and seafood, this is just one tiny part of it. The default
5-euro bento they bring to you in very standard Tokyo restaurants could
contain fish and meat of all kinds, but also sauces, spices, vegetables,
rice, soy, salad. Some parts are raw, some are deep-fried, some barely
cooked. I never knew tofu could be prepared to be so delicious. Rice is
always perfect. If you never tried, you owe it to yourself to try
<a class="reference external" href="https://en.wikipedia.org/wiki/Okonomiyaki">okonomiyaki</a> and <a class="reference external" href="https://en.wikipedia.org/wiki/Takoyaki">takoyaki</a>. We found a restaurant in Osaka
that serves takoyaki for a bit less than 2 euros for 6 pieces, when 12
pieces will fill you up. A ramen bowl served with half-boiled egg and
vegetables can serve as a single meal for a day. Soups are largely
different from one place to another, always delicious.</p>
<img alt="Food in Japan" src="images/japan-food1.jpg" />
<p>Finding a place to eat in Japan is usually not an issue. Everywhere you
look there are between three and thirty restaurants facing you. We tended
to stick to the ones offering menus with pictures, where we had a chance to
show what we wanted and have an idea of how much we were going to pay. Some
restaurants have an vending machine where you insert a note and press one
of the many buttons to choose your dish. A ticket gets printed, which you
hand over to the cook who will prepare it for you. Only trouble is: all
buttons are Japanese-only, so we usually went back and forth between the
window outside where plastic replicas displayed the available food, noting
the price down. If the dish of your choice was the only one costing 530
yens, you knew what you were having.</p>
<p>Even if Tokyo is a gigantic city, it does not have that oppressing feeling
you can have inside Paris, for example. Residential areas are packed with
small houses that have at most two floors, letting you see the sky all
around you. In business districts, skyscrapers are sufficiently far from
each other that you always see the sky, giving you the impression you can
breathe. Of course there are exceptions. Train stations and the quarters
around them tend to be real mazes of streets and hidden passages you can
wander aimlessly for hours on end without ever seeing the light of day.</p>
<img alt="Shibuya crossing" src="images/japan-shibuya.jpg" />
<p>We visited Shinjuku, a busy business district with office skyscrapers all
around, filled with salarymen. Around 8pm, we saw millions of them walking
towards the subway. Salarymen and women are equally dressed with deep blue
pants or a skirt, and a white shirt, wearing tired black shoes. When we
reached Shinjuku station we found a store selling salarymen uniforms:
endless lanes of deep blue pants and white shirts, with thousands of people
all dressed the same, looking carefully around to find their next deep blue
pants and white shirt. It looked so much like a cartoon. When in the
subway, they patiently queue where indicated, politely waiting for
passengers to come out of the cars before they rush in. Some of those cars
were so packed it was suffocating, and the subway usually runs every minute
or so. Who would drive a car in such a big city anyway when there are
trains to take you anywhere for pennies?</p>
<img alt="Shinjuku station" src="images/japan-shinjuku.jpg" />
<p>You might have heard of the cat cafés in Tokyo: yes they exist, together
with rabbit cafés and owl cafés, where you can have your tea and pet a
furry for an hourly fee. In Akihabara we saw those 8-floor buildings filled
with arcade games producing deafening noises, endless streams of fighting
heroes, car races, and cute little animals singing annoying songs at full
volume. We also visited manga cafés, manga buildings, figurine shops, and
duty-free electronic stores. We also got into sex-shops, more by accident
than anything. From the outside all you see are bright colors and posters
of female anime characters. Once inside all doubt is gone: the mangas are
not the kind you expected, and the large collection of DVDs is probably not
something you want to see because you won't be able to unsee it ever again.</p>
<img alt="Sex Shop in Tokyo" src="images/japan-sexshop.jpg" />
<p>The streets of the electric city are filled with gigantic screens
displaying ads, shouting slogans to the crowd as they pass by. There is no
escaping those jingles and loud voices trying to sell you stuff, making
<em>Blade Runner</em> feel like a documentary. At night, the lights are just
magnificent, the information flood fills you with a sense of panic which
makes me happy I cannot read. I wondered how we got through Shibuya without
inducing an epileptic seizure.</p>
<img alt="Akihabara" src="images/japan-akihabara2.jpg" />
<p>In Takeshita Dori, we were surrounded by school girls looking for kawaii
(cute) things to buy. Shops are filled with pretty much every conceivable
product covered with images of little large-eyed furry animals staring at
you like Bambi. The street is drowned in a thousand child songs playing too
fast, laughing girls, brightly-coloured clothes and people yelling about
their stores. I wish I could see that on a quiet winter evening.</p>
<img alt="Osaka" src="images/japan-osaka.jpg" />
<p>You always hear about the Japanese culture and how it wonderfully mixes
tradition and modernity, but you really understand what it means when you
find kimono-wearing twenty-something girls looking for the latest cameras
at Yodobashi, or when you see a tiny shrine on a street filled with bright
neon lights displaying manga characters. The working men and women of
Japan, all wearing uniforms, are facing a wave of creativity that has
little equivalent anywhere else in the world. You see a subway car packed
with tired salarymen who only want to get home, surrounded by cheeky
colourful adverts inviting them to take better care of their skin or why
not travel to a distant country. Cartoon characters showing their butts,
movie stars dressed as animals, and dancing pokemon figures are just
expressions of pure creativity. Stark contrast.</p>
<img alt="Pokemon" src="images/japan-pokemon.jpg" />
<p>Three weeks and a million anecdotes later, I realize some things have
changed in me and I don't quite know what yet. Taking away some of the
conventions I have unknowlingly applied all of my life and adding some
more, gets me a little closer to understand what being human and living in
society really means. Three weeks of being illiterate is a humbling
experience.</p>
What time is it?2018-07-09T22:39:00+02:002018-07-09T22:39:00+02:00nicolas314tag:maz.one,2018-07-09:/what-time-is-it.html<img alt="clockface" src="/images/clock.png" style="width: 262px; height: 300px;" />
<hr class="docutils" />
<ul class="simple">
<li>Hello Mr. Engineer, can you tell me what time it is?</li>
<li>No I can't.</li>
<li>Why?</li>
<li>Well then. You see, my watch is an electronic and mechanic device based on the oscillation of a quartz that imprints a periodic movement to a set of cogs, which are then de-multiplied to lower …</li></ul><img alt="clockface" src="/images/clock.png" style="width: 262px; height: 300px;" />
<hr class="docutils" />
<ul class="simple">
<li>Hello Mr. Engineer, can you tell me what time it is?</li>
<li>No I can't.</li>
<li>Why?</li>
<li>Well then. You see, my watch is an electronic and mechanic device based on the oscillation of a quartz that imprints a periodic movement to a set of cogs, which are then de-multiplied to lower the base quartz frequency from 32,758 Hz to exactly 1 Hz, i.e. one beat per second.</li>
<li>That's very nice. And what time does your watch show now?</li>
<li>I could tell you but it would not be useful. See, the quartz frequency is not exactly that power of two, it is itself oscillating with a larger period around that value, meaning that my watch can be ahead or behind by some amounts that are hard to measure, let alone predict.</li>
<li>So it is inaccurate?</li>
<li>Yes! You can never tell exactly the time with that kind of device.</li>
<li>Ok... Seriously, what time is it?</li>
<li>Not only are the watch mechanics imprecise, but they do not take relativistic effects into account.</li>
<li>That so?</li>
<li>Yep. Since Einstein we know time is nowhere absolute. When I put my arm up like this, time flows a little slower because of the Earth rotation, and if I put it down like this is goes a bit faster. Or is it the other way around? Anyway, my time reference is unlikely to be the same as yours since we are not moving around in sync.</li>
<li>Listen, this is all very nice but that was not my question. Will you tell me the time it shows now and I will deal with the imprecision myself?</li>
<li>No can't do.</li>
<li>Why is that?</li>
<li>Even if you discard all relativistic effects and frequency drifts, the notion of time is not something universal on Earth.</li>
<li>Care to explain?</li>
<li>Time is only valid in a given time zone. Since the end of the 19th century we have split world regions according to time zones which keep changing at regular intervals based on political choices. In order to be able to tell you the time of day, I need to know a reference time in a given place and convert that depending on your position on the planet. We could use GMT, which stands for Greenwich Mean Time, but it is not even indicating the current time in Greenwich UK. I could then program a microservice that could give you the current date/time based on an estimated position from your IPv4 address, provided you are not too close to a time border. But then that assumes you have Internet access. Oh wait, do you have an iPhone or an Android?</li>
<li>Er... Thanks mate. So let's say we use the current time zone, Ok?</li>
<li>Do you know if we apply Daylight Saving Time where you stand?</li>
<li>How would I know? Yes, probably!</li>
<li>Probably with what probability? Because we could weigh the answer depending on... Hey, where are you going?</li>
<li>To lunch. I just remembered I wanted to ask you if it was time for lunch.</li>
</ul>
Counting Camels2017-11-01T23:37:00+01:002017-11-01T23:37:00+01:00nicolas314tag:maz.one,2017-11-01:/camels.html<img alt="Camels" src="/images/camels.png" style="width: 35%;" />
<p>I read somewhere in a math history book that numbers were actually invented to count camels. Someone wanted to send over a herd of camels to be sold on a market on the other side of the desert and they did not trust the camel escort. How would the receiving …</p><img alt="Camels" src="/images/camels.png" style="width: 35%;" />
<p>I read somewhere in a math history book that numbers were actually invented to count camels. Someone wanted to send over a herd of camels to be sold on a market on the other side of the desert and they did not trust the camel escort. How would the receiving party know if some camels had not been stolen on the way? So they used a fairly simple principle: line up your camels, put one pebble in front of each. Gather the pebbles, put them in a small jar, burn the cork, hand it over to the escort.</p>
<p>On the receiving end, break the jar, put one pebble in front of each camel. You will know immediately if camels are missing.</p>
<p>This apparenly went on for a while, until someone figured out that instead of lining up pebbles and camels you could shorten the process by writing signs on the jar to indicate how many pebbles were inside. On the receiving end you just had to look at the signs and compare to what you saw. In case of doubt, break the jar and line up pebbles and camels. And then it was just a matter of time until somebody noticed you don't need the pebbles and the jar. Just cook a clay tablet in an oven with a text indicating how many camels you are sending.</p>
<p>I have no idea if this story is true or not, but I like the way it stresses the breakthroughs that have happened. Going from a bijection pebbles/camels to a bijection in camels/signs was brilliant. I expect the first attempts were likely to just draw plain strokes on the jar, as many as there were camels in the herd. The next breakthrough was simplifying a whole bunch of strokes into a single sign, e.g. using a hand to signify the number 5. And the last one was to realize that the jar and pebbles were unneeded.</p>
<p>Another shift that amazes me to this day is how money actually works. The first currency tokens had actual value, they were made of metal you could melt and use if you so wanted. When the first bank notes were introduced, they switched from actual value to a potential: the note said that you could obtain real metal if you were to exchange that note in a bank.</p>
<p>We now live in a world where I can pay my lunch by waving a piece of plastic over a radio-equipped terminal connected to a bank. My plastic contains numbers that cannot be found on any other credit card, which are used to authenticate me. Now my bank makes a promise to pay my meal to the restaurant's bank. No metal or paper changes hands.</p>
<p>Since a few years, things are shifting again. Instead of waving a credit card containing my unique account identification numbers, I can now use a mobile phone that contains a series of numbers that are only valid for myself, my account, for today, and for limited amounts. This is what they call tokenization and the reason it is booming is that it is a lot simpler to store temporary tokens with limited value than long-term banking credentials with unlimited powers. Security needs not be that high, though you still need to be able to authenticate account owners in a very secure way, but there are plenty of ways to achieve that.</p>
<p>Among the strongest methods we know today to authenticate someone, the most popular relies on the fact that you cannot split a big number into a multiplication of primes. If you tried with a gigantic computer, it would require more heat to power than is available in the universe.</p>
<p>We have come a long way since camel-counting.</p>
I am not your daughter2017-10-02T09:42:00+02:002017-10-02T09:42:00+02:00nicolas314tag:maz.one,2017-10-02:/i-am-not-your-daughter.html<img alt="Sleepless" src="/images/sleepless.png" style="width: 512px; height: 448px;" />
<p>You called me quite late. Some time during the middle of the night, and for
whatever reason I had left my phone on. I picked it up and was greeted by
your anxious voice:</p>
<div class="line-block">
<div class="line"><em>- Isabelle, is that you?</em></div>
</div>
<p>Part of my brain was still actively dreaming, but the part that …</p><img alt="Sleepless" src="/images/sleepless.png" style="width: 512px; height: 448px;" />
<p>You called me quite late. Some time during the middle of the night, and for
whatever reason I had left my phone on. I picked it up and was greeted by
your anxious voice:</p>
<div class="line-block">
<div class="line"><em>- Isabelle, is that you?</em></div>
</div>
<p>Part of my brain was still actively dreaming, but the part that was
emerging found the idea preposterous. Do I sound like an Isabelle? I
croaked:</p>
<div class="line-block">
<div class="line"><em>- No Madam, this is not Isabelle.</em></div>
<div class="line"><em>- Oh come on Isabelle, it's Mum. Stop playing fool with me, I recognize your voice, please..</em></div>
</div>
<p>Even half awake, I definitely noticed the tension in your voice. You wanted
to talk to your daughter and nothing would stop you.</p>
<div class="line-block">
<div class="line"><em>- Madam, I can assure you I am not your daughter. In fact I am a man and my name is Nicolas.</em></div>
<div class="line"><em>- Bullshit! Isabelle, talk to me!</em></div>
</div>
<p>Your old lady's tone left me no choice but to obey, so I gave up and decided to play along.</p>
<div class="line-block">
<div class="line"><em>- Alright Mum, you got me. What's up?</em></div>
</div>
<p>You seemed surprised. Apparently Isabelle does not give up so easily when
playing this game. But the sudden joy of being able to talk to your
daughter was so great that you could not help it. You started talking about
your neighbours at the retirement home, how the nurses were treating you,
and had many complaints about the food and such. I listened very carefully
at first and quickly dozed off, we were half way during the night after
all.</p>
<p>You called again a bit later, and again, and again. We spent our night like
this: you finally talking to your daughter, and me sleeping through
10-minute intervals. Finally one of your nurses must have found out you
were secretly phoning at night and you stopped calling.</p>
<p>You never called again. I hope you found the right number for Isabelle and
she takes good care of you.</p>
Long live NAT!2017-02-28T23:41:00+01:002017-02-28T23:41:00+01:00nicolas314tag:maz.one,2017-02-28:/long-live-nat.html<p><img alt="ipv6-no-thanks" src="/images/ipv6-no-thanks.png" style="width: 60%;" />Home networking can be a lot of fun: setting up a name
service, a guest network, or traffic rules, leads to an endless joy of
discovering new RFCs or creativity in the very active field of artistic
configuration file syntax.</p>
<p>I thought I had seen everything until I tried to …</p><p><img alt="ipv6-no-thanks" src="/images/ipv6-no-thanks.png" style="width: 60%;" />Home networking can be a lot of fun: setting up a name
service, a guest network, or traffic rules, leads to an endless joy of
discovering new RFCs or creativity in the very active field of artistic
configuration file syntax.</p>
<p>I thought I had seen everything until I tried to set up IPv6 connectivity
for my home network. Little did I know that this would eat up so many of my
precious free evenings. The following writeup is here to remind me never to
try that kind of shit ever again, and as a warning to future generations
who might want to dig into this kind of topic. Life is short, there are
many better things to do than attempt to set up a new addressing scheme for
your home network. Long live the NAT king!</p>
<div class="section" id="the-start">
<h2>The Start</h2>
<p>It all began when I noticed that my ISP provided me with a unique (native!)
IPv6 prefix to use on my home network. Something like:</p>
<pre class="literal-block">
2001:1234:5678:9abc::/56
</pre>
<p>Since I was not familiar with IPv6 addresses, it took me a while to find
out that the first 64 bits of a 128-bit IPv6 address designate a network,
and the last 64 are reserved to differentiate hosts on that network. My
provider handing me a /56 means I have 64-56 = 8 bits to play with, i.e. I
can instantiate 256 home networks, each having up to 2^64 =
18,446,744,073,709,551,616 hosts. Overshot a bit, maybe.</p>
<p>So where do I start? Do I have to install specific software? Where? Do I
need to buy specific hardware? How many services are needed? And thereby
started my long painful descent into the horrific world of IPv6. Toss and
loose 1d20 sanity points immediately.</p>
<p>My ISP unfortunately did not provide any help as to what I am supposed to
do with the IPv6 thingie they gave me. No single help page, very few
discussions on their forums, and all exchanges I had with customer service
were completely useless. Best I could find were discussions between
customers of an ISP in the US that provides a similar setup. That is thin.</p>
<p>Say you received a /56 prefix from your ISP. If that prefix ever changes
e.g. because you switched to a new ISP, you want things to work
automagically because that is the way things currently work with IPv4:
changing my public IPv4 address does not change anything to my home
network.</p>
<p>In order to do that, IPv6 suggests that home networks use two sets of
addresses: the public ones derived from the ISP-provided /56, and another
private address space based on something else called a ULA (Unique Local
Addresses). You get to choose your own ULA on your home network(s),
preferrably based on a good random number generator, but nothing prevents
you from taking something like fc00:caca:caca:caca:caca::/48. If anybody
else on the Internet picks the same network prefix you will get into
trouble when trying to get intimate with each other, e.g. by establishing a
VPN between both worlds. We had exactly the same problem when trying to
join two sites using IPv4 NAT'd 10.0.0.0/8 subnets, so this is not really a
regression. Fun fact: if you have no ULA in France you can always say <em>"Il
manque ULA sur mon réseau"</em>.</p>
<p>How do you get to choose this ULA? If you happen to have a single router on
your home network it should just be a matter of digging through the router
IPv6 setup until you find it. But most home networks are now running
multiple routers that are all unaware of each other, and all convinced they
are masters of the universe. You will most certainly end up with several
ULAs. Some of your devices will get several addresses and you will have to
understand your own network topology to know which address to use to access
them. Prepare for glorious hours of debugging, which is particularly great
when facing addresses that are mostly made of bloody random bits.</p>
<p>Why several routers on the same home network? Simply because you may be
running several DSL connections, or maybe you have a VPN started somewhere
away from your edge router, or maybe you connected your smartphone and it
offers another potential exit to the Internet. You also get a virtual
router when you start virtual machines on a desktop.</p>
<p>To make things <em>simpler</em>, every network interface on your machines will
also generate a local address that is only valid for its closest
neighbours, called a link-local address. Unfortunately you won't go far
with that one as it is not supposed to cross boundaries. Think of it as a
127.0.0.1 that extends to the other side of the cable but not further.</p>
<p>Ok so we have now several adresses for each machine on the network.
Figuring out which one should be used (incoming or outgoing) is just an
unspecified, incredible mess. The link-local address can only be used on
very specific physical links, the ULA address cannot be routed to the
Internet, and the public addresses you have may change at any moment, e.g.
through your smartphone sharing a 4G access.</p>
<p>At that point we have just determined that your printer currently
identified as 'printer' also known as 192.168.1.20 in IPv4 will now be
accessible as:</p>
<div class="line-block">
<div class="line">- <tt class="docutils literal">fe80:bffa:3d5f:5f8d:b4cf:1749:b01c:5b2f</tt> for machines directly connected to it through an Ethernet cable</div>
<div class="line">- <tt class="docutils literal">fc00:c465:3b76:b34d:38f7:da19:2586:1cbd</tt> for machines living on the same internal network.</div>
<div class="line">- <tt class="docutils literal">2001:61af:ff44:b148:4fc3:0097:f35d:c806</tt> for machines on the internet when reached through a first ISP, and another public address for each available ISP connection.</div>
</div>
<p>Oh joy.</p>
<p>Of course normal human beings are not meant to remember this kind of random
shit. For this kind of thing you have DNS.</p>
</div>
<div class="section" id="dns-you-said-what-dns">
<h2>DNS you said? What DNS?</h2>
<p>There are really two ways machines can obtain an IPv6 address: SLAAC and
DHCPv6. SLAAC means Stateless Address Auto Configuration, whereby a machine
obtains a prefix and derives its own IP address from it, e.g. based on its
own MAC address. Cool, right? You do not have to assign individual
addresses in static DHCP leases, every machine does it on its own. But
then: how do you know which address was self-assigned by your very smart
printer?</p>
<p>There are dedicated neighbour-discovery protocols for that, but they are
mainly designed to make sure that addresses are locally unique and routers
know where to find them. This is only taking care of establishing a link,
there is nothing dedicated to associating a name to a self-assigned IP
address. And if there was, how would you know who to believe? If two
machines on the local network claimed to be 'joe', what should happen?</p>
<p>To be fair, there are solutions like Bonjour, also known as zeroconf, but
they are unlikely to work on lightweight or old devices. Shoot again.</p>
<p>Back to square one: if you want to reach your own machines using
human-usable names you need to run DHCPv6, a protocol that was designed to
compensate for such things. And there you go: back to static leases,
addresses assigned by a router, attached to a name, and you end up doing
exactly the same kind of shit you used to do with IPv4 local networks,
except this time the addresses are much easier to screw up.</p>
<p>Even worse: if the self-assigned IPv6 addresses are not related to MAC
addresses, it means every single host on your local network will have
generated its own random address, forcing you to manually harvest them from
all devices. But you know how to do that on your connected toaster, right?</p>
<p>What's in it for the average home network user? Pretty much nothing. The
fact that every single one of your home devices has a potentially reachable
address on the intertubes is downright scary. Internet service is for
internet servers, not for sensors and other IoT bullshit. First thing you
will want to do is bullet-proof your firewall to make sure nobody but you
can access your printer from the Internet, and hope things are Ok with your
IoT shit.</p>
<p>The story did not just end up with me reading thousands of pages on the
Internet and a couple of paper books. I hacked every single computer in my
house to run IPv6, starting with the routers under OpenWRT, LEDE, FreeBSD,
OpenBSD, pfSense, OPNSense, and later moving on to all client OS machines:
OSX, Linux, Android, *BSD, and even some Windows boxes, blimey. I
instantiated dedicated DHCP and DNS servers, configured static addresses,
automatic ones, bridges and NATs and firewall rules and what-have-you, and
I ended up with some machines working under IPv6 only, some under IPv4
only, some that could use both stacks, and some (a lot) that were just
unreachable no matter what. Yeah, I also crashed my Internet access several
times. Omelet and eggs.</p>
<p>Let me try to put it this way: some of my home machines are servers, e.g. a
NAS or a printer. I want to be able to print on 'printer' or mount a share
on 'NAS' without having to remember random 128-bit numbers. Silly me. Since
I want to use names I have to assign addresses myself from a router running
DHCPv6. Neither NAS nor printer need to be available to the public. So what
did I gain compared to a local IPv4 network? Hmm... Address management is
not fun with 32 bits, imagine with 128.</p>
<p>Or maybe I am just old-fashioned, trying to manually assign names to my
home machines. This might be an idea for a new product: a router that would
automatically identify hosts on the home network and show them on a single
web interface, allowing you to assign names and forget about addressing
altogether. Might get in trouble when you have several identical devices
but I'm sure there would be a way. If such a product exists I have not seen
it yet.</p>
<p>On the other hand, if I want to browse the Interwebs in v6, I found out
that mounting a SOCKS proxy on a remote cloud box works perfectly well. No
need to configure anything, just ssh -D and the IPv6 world is mine to
browse.</p>
</div>
<div class="section" id="summing-it-all-up">
<h2>Summing it all up</h2>
<p>Address assignment is not easier than IPv4. Still requires a dedicated DHCP
and DNS server, only more complicated to configure. You are facing the
tedious task of gathering self-assigned IPv6 addresses from all hosts and
copying them onto your DHCPv6 server, hoping the self-assignment method
won't change soon.</p>
<p>Routing is now different, but not easier. New constraints are imposed on
knowing which interface to bind to when reaching out to the Internet.</p>
<p>Firewalling the whole thing with a mix of IPv4 and IPv6 might tear you a
new one. I can already lock myself out of a router with human-readable
firewall rules, I cannot imagine doing the same thing with batshit-crazy
addresses and feel safe.</p>
<p>You know what? I will stick to glorious NAT'ing until this mess is sorted
out. Good news is that there are many bright people currently working on
the topic. All I hope is they eventually come up with something that you
and me can use without having to read through a million pages of RFCs,
compile obscure daemons, or purchase new boxes as if I did not have enough
of them.</p>
<p>Talking about RFCs, this one is trying to gather very sensible requirements
about home networks:</p>
<p><a class="reference external" href="https://tools.ietf.org/html/rfc7368">RFC 7368</a></p>
<p>If you have 20 minutes to spare, you should watch this talk:</p>
<p><a class="reference external" href="https://www.youtube.com/watch?v=wQdfWUsG4uI">Routing IPv6 in the Homenet</a></p>
<p>If you really insist on switching your home network to IPv6, I would recommend reading this rant first:</p>
<p><a class="reference external" href="http://www.kloepfer.org/ipv6-homenet.html">IPv6 at home (published 2012, still relevant)</a></p>
<p>In its current state I can only dismiss the current IPv6 definition for home networks as very incomplete and unworkable for non-professionals. Let's hope RFC 7368 will be handled by qualified, creative, and pragmatic people.</p>
<p>Til then, there is no place like <strong>127.0.0.1</strong></p>
</div>
Wunder Weather2016-08-16T13:46:00+02:002016-08-16T13:46:00+02:00nicolas314tag:maz.one,2016-08-16:/wunder-weather.html<img alt="wunder logo" src="/images/wunder.png" />
<p>Just released this small piece of code a few days back:</p>
<p><a class="reference external" href="https://github.com/nicolas314/wunder">https://github.com/nicolas31/wunder</a></p>
<p>I wanted to be able to bring up the weather forecast for the place I am currently visiting without having to yield my address book to a shady app, or suffer from tons of …</p><img alt="wunder logo" src="/images/wunder.png" />
<p>Just released this small piece of code a few days back:</p>
<p><a class="reference external" href="https://github.com/nicolas314/wunder">https://github.com/nicolas31/wunder</a></p>
<p>I wanted to be able to bring up the weather forecast for the place I am currently visiting without having to yield my address book to a shady app, or suffer from tons of annoying ads eating through my data plan and phone storage.</p>
<p>The Yahoo weather app is fantastic but has too many ads. Weather web sites are incredibly data heavy, making it nearly impossible to get right to the information I am looking for: is it going to rain today or tomorrow?Expected temperatures? Android has some ad-less widgets but they usually request GPS positioning and I'd rather not activate location services when I don't need them.</p>
<p>So I hacked something. Made a web app that identifies your position by geolocating the requester's IP address, obtains the weather forecast from a reliable source, and displays the only weather information I need on a fast loading page.</p>
<p>First issue: geolocate an IP address.</p>
<p>There are many free services on the net to achieve that. Alternatively, you can download a static list and refresh it at regular intervals, but I wanted to get something a bit more dynamic. I chose:</p>
<p><a class="reference external" href="https://ip-api.com">https://ip-api.com</a></p>
<p>Their API is dead simple and just works. Provide an IP address, get a country code, city name, latitude and longitude. You do not need to subscribe to their services, just make sure you are not choking them with too many requests.</p>
<p>Second issue: find a reliable weather source.</p>
<p>I fist tried<a class="reference external" href="https://www.openweathermap.org">openweathermap.org</a>. This is a very cool site but has a few shortcomings:</p>
<p>You can get the weather for a given [city, country] or [lat, lon]. The list of supported [city, country] pairs is static and can be downloaded from their web site. While they do support a lot of cities in the world, the problem was figuring out how to match [city, country] between what is returned by ip-api.com and what is understood by openweathermap.org. The matching is not 100% accurate.</p>
<p>Getting the weather by coordinates would work but it is far from user-friendly. You end up with Weather forecast for location Lat=XX Lon=YY. I'd rather look up the weather for San Francisco than for a pair of coordinates that are not obviously recognizable.</p>
<p>I ended up looking up [city, country] by computing the smallest distance on the openweathermap list, but that is just tedious and a lot of work for very little gain.</p>
<p>Other major issue: the weather forecast is only provided GMT, which is utterly useless. What I want is local time, always. What do I care if I am told that it will rain from 2 to 5am GMT if I cannot relate that to local time?</p>
<p>Figuring out a conversion between GMT and local time is a lot trickier than it looks. Thanks to Daylight Saving Time rules that are changed at random intervals in various countries, it is very hard to predict the time offset in some places more than a couple of weeks ahead. Relevant:</p>
<ul class="simple">
<li><a class="reference external" href="https://mm.icann.org/pipermail/tz/2016-July/023855.html">Egypt abolishes DST (July 2016)</a></li>
<li><a class="reference external" href="https://en.wikipedia.org/wiki/Daylight_saving_time_by_country#/media/File:DST_Countries_Map.png">Daylight Saving Time by country as of July 2016</a></li>
</ul>
<p>A bit of googling around revealed there is an actual API from Google Maps to convert a Unix time stamp + latitude and longitude to a local time. This API takes into account local DST rules at the considered date/time, which is exactly what we want. No need to register with Google, as usual the API is free to use and rate-limited.</p>
<p>Example code can be found here: <a class="reference external" href="https://github.com/nicolas314/tz">https://github.com/nicolas314/tz</a></p>
<p>In summary: getting the weather from openweathermap would require:</p>
<ul class="simple">
<li>One external API call to associate IP to [lat, lon]</li>
<li>A search to associate [lat, lon] to [city, country]</li>
<li>One external API call to obtain actual weather data</li>
<li>One external API call to convert GMT to local time</li>
</ul>
<p>I have implemented that and the result is ugly. Ok let's see if we can find something smarter.</p>
<p>Next try:<a class="reference external" href="https://www.wunderground.com">wunderground.com</a></p>
<p>They also offer an API to obtain weather data for any place in the world and they take care of two things: converting [lat, lon] to [city, country], and converting weather forecast to local time. This is exactly what we want.</p>
<p>Their API can also take care of geolocating an IP address but I found their results to be a lot less reliable than what I get from ip-api.com, so will stick to that for geolocation.</p>
<p>Their terms and conditions are fair. You need to register with them to obtain an API key and that's about it. Results are delivered in metric units and can be localized in several languages. You also get a pointer to icons symbolizing the weather, which is perfect to generate a nice web page effortlessly.</p>
<p>Some comments about my implementation:</p>
<p>Results from wunderground contain a whole bunch of information I am not interested in, like temperatures in Farenheit. Not an issue: the Go JSON API allows defining fewer fields than what is parsed, so you can keep your structs small with only relevant data.</p>
<p>When running behind a reverse proxy, the incoming requesting IP address you see is the one for the proxy. In order to get the real incoming IP address you need to configure the reverse proxy to pass it along, usually in an HTTP header. Since I am running this service behind nginx, I get the address from X-Real-IP. That is probably different for each reverse proxy out there.</p>
<p>Hardcoded handlers are provided to take care of requests for /favicon.ico and /robots.txt. I was tired of seeing 404 requests in my logs for these two.</p>
<p>Results are cached by IP address for one hour to avoid flooding upstream API services with requests. Results are displayed from a template that can easily be tweaked. The one I wrote fits nicely enough on both mobile and desktops, your mileage may vary.</p>
<p>I installed the end result on a tiny VPS instance, for my own use. Hoping that could be useful to somebody else.</p>
Printers from Hell2016-08-12T16:25:00+02:002016-08-12T16:25:00+02:00nicolas314tag:maz.one,2016-08-12:/printers-from-hell.html<img alt="A printer" src="/images/printer-clipart.png" style="width: 50%;" />
<p>My first printer was a black-and-white Samsung laser device that caught my eye in a brick-and-mortar shop by sporting a brave Linux sticker on the front side. These were the early 2000s, mind you, and finding hardware that openly advertised Linux compatibility was quite unusual. The same sticker also showed …</p><img alt="A printer" src="/images/printer-clipart.png" style="width: 50%;" />
<p>My first printer was a black-and-white Samsung laser device that caught my eye in a brick-and-mortar shop by sporting a brave Linux sticker on the front side. These were the early 2000s, mind you, and finding hardware that openly advertised Linux compatibility was quite unusual. The same sticker also showed a Windows and Apple logo, to be honest.</p>
<p>Linux compatibility in those times meant that you were a mere 37 friendly steps away from printing a page, which is quite an achievement if you remember that Linux printing in those ages was still reserved to a very small number of gurus worldwide. You needed to re-compile your kernel, install the correct kind of USB support, a variety of device drivers, a couple of daemons, a postscript interpreter, some fonts, and all the extra cruft that comes bundled with those packages. Under Debian this goes as a simple apt-get but RPM-based distros were less automated. You had to dig through dependencies yourself and keep throwing packages at your hard drive until the damn thing stopped complaining. Tedious. And then you could finally start editing the printer configuration files. If you have never configured these horrors, think of a mix between sendmail.cf and procmailrc.</p>
<p>Since then CUPS has changed the game. Config files are just as obscure but are now XML-based (now you have two problems). CUPS forces you to assign network permissions to access your very local USB device, requiring at least some knowledge of basic network security to be able to print a page. You also had an option to configure CUPS through a web interface, but that also required manipulating some networking rules and authorizations in an XML file to just bring up the page. Oh the joys of manual CUPS configuration!</p>
<p>Cue to 2007: I purchased a combined color printer and scanner from HP for close to nothing, hoping I would not need the ink. Instead of hooking it into a Linux box, I decided to take the easy path and attach it to a Mac. This is still running CUPS but at least things are a bit automated with Apple: click "Add Printer" and follow the steps until it says it works. Surprisingly enough, you needed to delete and re-add the printer every time you upgraded the OS, but thanks to the provided wizard this was not such a pain.</p>
<p>This HP stuff did not work so well. It printed alright, but the cartridges seemed to empty at lightspeed. The scanner was incredibly slow, it could not be controlled from any other machine than the one connected through USB. Better than nothing, I suppose.</p>
<p>Cue to 2014: I saw this fantastic discount for a multi-function color printer and scanner from Epson, for a mere 40 euros. That is about the price of ink cartridges for other printers. Count me in! I had the thing delivered to my place the next day.</p>
<p>Without having read any of the documentation I tried to get the printer to work while attached to a Windows PC. Several hundred megs of software were downloaded, installed, configured, and the results were appalling. The printer would not connect half of the time, I managed to print a test page and that was it.</p>
<p>And then I saw the WiFi logo on the box. WiFi? Sure enough, there was also an RJ45 plug on the back. Still armed with my best pioneer spirit, I painfully configured a WiFi connection on the printer itself -- on a tiny 2-inch screen -- and lo and behold: the printer became immediately available on the local network for everyone's enjoyment! Seems this time they stuffed the complete CUPS layer directly inside the printer, and it actually worked! Oh wow.</p>
<p>Ok, it does not always work. You still need to delete a re-configure the printer every now and then but it has become a lot less painful than kernel re-compilation or XML editing.</p>
<p>I really can't complain. Printing now works from pretty much any machine at home without having to install a metric ton of crapware and keeping it updated. The same machine also offers a "Scanning to the cloud" option that sometimes works. It seems every document I scan for myself has to be first sent to China for approval before it can reach my email or my Dropbox account. Just don't scan stuff when the Chinese guy is having a cigarette pause. As a better option, I put a USB stick into the printer, scan and save onto it, and manually carry the stick back to a computer. And then I lose the USB stick, and it takes me ages to figure out where it is, but that part is only my fault.</p>
<p>I was not so lucky at work. Configuring a printer on a laptop is still as cumbersome as ever. To be fair, I have never seen a printer configuration task use less than three experienced engineers to achieve. Repeat every time the document you want to print is important and on a tight deadline. In many cases I completely gave up configuring a work scanner. Life is too short.</p>
<p>The time is 2016 and we still have not figured out how to print and scan easily in a standard household or office. As a friend of mine recently told me: "If we ever reach the Singularity one day, we'll just ask the AIs to configure a printer. That should buy us enough time to invent spaceships and leave the planet before they enslave us all."</p>
<p>See also:<a class="reference external" href="https://theoatmeal.com/comics/printers">https://theoatmeal.com/comics/printers</a></p>
OpenWRT on Ubiquiti AC Lite2016-05-30T16:41:00+02:002016-05-30T16:41:00+02:00nicolas314tag:maz.one,2016-05-30:/openwrt-on-ubiquiti-ac-lite.html<img alt="Ubiquiti Unifi AC-lite" src="/images/unifi-ac-lite.jpg" style="width: 262px; height: 300px;" />
<p>A year ago, I thought I had upgraded my home WiFi to AC with the purchase of a cheap TP-Link Archer C5, but it only gave me trouble. The 2.4GHz band is working perfectly, the 5GHz not so. The first version of OpenWRT I installed had no support at …</p><img alt="Ubiquiti Unifi AC-lite" src="/images/unifi-ac-lite.jpg" style="width: 262px; height: 300px;" />
<p>A year ago, I thought I had upgraded my home WiFi to AC with the purchase of a cheap TP-Link Archer C5, but it only gave me trouble. The 2.4GHz band is working perfectly, the 5GHz not so. The first version of OpenWRT I installed had no support at all for the 5GHz mode, I had to dig out from the openwrt mailing-lists that something was under way to add support for it, find the right rxxxx release, compile it myself and install it, only to be disappointed. In the end I got it <em>almost</em> working: network would drop every minute or so, and the range would not exceed a few meters away from the access point. Quite worthless.</p>
<p>Ars Technica reported last year about switching from consumer-grade WiFi access points to professional ones here:</p>
<p><a class="reference external" href="http://arstechnica.com/gadgets/2015/10/review-ubiquiti-unifi-made-me-realize-how-terrible-consumer-wi-fi-gear-is/">Ars Technica review of Ubiquiti Access Points</a></p>
<p>Took me a while to finally give up on the Archer C5 and decided to get the cheapest Access Point from Ubiquiti: <a class="reference external" href="https://www.ubnt.com/unifi/unifi-ap-ac-lite/">the Unifi AC Lite unit</a>.</p>
<p>Several things play in favour of this access point: it is designed to be hanging on a wall or a ceiling for a better wave spread, offers much higher power than your usual consumer-grade WiFi device, and the firmware is in the hands of Ubiquiti so they are taking care of making sure the unit is performing as it should. Or so I thought.</p>
<p>First surprise: the device cannot be started out of the box. You need to download the Unifi Controller software, a 200MB piece of java software that is meant to control the unit. Install the Controller on any computer on your local network, start it up, let it discover your access point, and you are good to go. The Controller starts up a local web-based interface which is accessed via a local web browser.</p>
<div class="line-block">
<div class="line">Ubiquiti designed it this way because this product is not meant to be purchased as a single unit but in batches and installed in hotels or offices. The Controller software has obviously been designed to maintain a long list of such access points. Having just one looks a bit ridiculous but Ok, I'll play along.</div>
<div class="line">To be fair: if you do not want to install the Controller locally you can also run it from an internet box (look Ma, it's the cloud!). For companies or hotels who do not have servers on site it is an excellent idea but for a single AP this is largely overkill. I am <strong>not</strong> keeping a java server up full time on the Internet to manage an Access Point in my living-room. And if you do not want to run the Controller at all, you can also just start it whenever you need to modify your AP configuration and shut it down afterwards, which is what I did.</div>
<div class="line">I mounted my AP on a top shelf, ran the initial configuration, and was delighted to see a double-band WiFi emerge immediately. Strong signal everywhere in the house, no connection issues, great! A welcome enhancement to my home network.</div>
</div>
<p>The only notable modification I brought was to stick a white Apple sticker on the Ubiquiti logo to hide it. The point was not to rebrand it as an Apple product but to hide this ugly U and this was the only sticker I had available that day. The Ubiquiti guys are probably not aware that they have the very same logo as a cheap French supermarket brand and I got tired of seeing that prominently displayed in my living-room.</p>
<p>After about a month, a few things started bothering me though. The AP seemed to have trouble waking up after losing power. First boot would report everything fine but WiFi would drop every connection immediately. Rebooting the AP solved it every time. Did not feel too good about this.</p>
<p>Nmap'ing the unit revealed an open ssh port, which accepted the admin credentials that were set on the controller software. Once logged in, I found myself in front of some kind of heavily modified OpenWRT. Interesting... So is there anything on the Ubiquiti web site about this modified OpenWRT? After all, <a class="reference external" href="https://wiki.openwrt.org/about/license">OpenWRT is under GNU license (v2)</a> so I expected to find sources, some kind of build system, or anything related to the modified OpenWRT version running on my Access Point, but I could not find anything, at least nothing obvious from the Ubiquiti web site. Bad point for Ubiquiti but I am not a lawyer.</p>
<p>Nosing around the logs for some explanation for the needed reboots, I found nothing obvious. What I found was that a process was spitting a few lines every five seconds about having no contact with the Ubiquiti Controller software. Several questions on the Ubiquiti forums on that topic were answered with: <em>"yes we know, this is a low-priority fix, just ignore it for the moment"</em>. I have to say that just pushed me over the edge. I understand that my case is not the general use case and I truly do not blame Ubiquiti, but this is not the way I want my AP to work. I do not care much about log files filling up with useless messages but the lack of interest for single-AP users like me is disturbing.</p>
<p>So off to installing a real OpenWRT firmware this time. I finally got it working but it took me a whole afternoon of research to do so, which is summarized here in case it can be useful to somebody else.</p>
<p>First: the current (3.7) firmware embeds an RSA signature check preventing any attempt to install non-Ubiquiti firmware. This is probably due to the recent FCC firmware lockup rules. While I do understand the FCC concerns and the important role they play in Homeland Security, I regretfully do not feel obliged to follow US rules on European ground. This is my hardware now, if I decide to mess it up with another firmware I should be able to do so.</p>
<p>Solution: downgrade the firmware to version 3.4, which is signed by Ubiquiti and does not check firmware update signatures. This firmware can be found by googling a bit around, I got a working URL and the complete procedure from this page:</p>
<p><a class="reference external" href="https://pwassi.privatedns.org/openwrt/unifiac/">LEDE/OpenWRT for Ubiquiti UniFi AP AC (LITE + LR + PRO)</a></p>
<p>While we're at it, it may be a good idea to switch from OpenWRT in favour of the recent project fork called <a class="reference external" href="https://www.lede-project.org/">LEDE project</a>. Seems they have added official support for this hardware and the documentation seems a lot cleaner, though very, very incomplete for the moment, which is perfectly Ok for a two-week old project.</p>
<div class="line-block">
<div class="line">If you did not follow what happened to the OpenWRT project recently, you may be interested in learning a bit more about why the team forked:</div>
<div class="line"><a class="reference external" href="https://lwn.net/Articles/686767/">https://lwn.net/Articles/686767/</a></div>
</div>
<p>Some pre-built images are available from the LEDE project site, but I chose to go all the way and clone the github repository, configure the build to include the software I need, and recompile everything myself. On a beefy x64 server this took about 2 hours and 11GB of disk space, ending up with a 3.3MB image that was happily installed in a single command on the downgraded Access Point.</p>
<p>The default configuration is smart: the AP tries to obtain an address for itself through DHCP on its only wired interface and acts as a bridge for wireless clients, making it immediately operational when connected to a network equipped with a proper DHCP server. Sweet.</p>
<p>Net result: my Unifi AC AP is now completely stand-alone. I happily removed the Ubiquiti Controller software and customized my AP to death with various scheduling and logging scripts. Wireless range in the 5GHz band covers the whole house and the 2.4GHz allows me to walk outside during Skype calls without losing signal. No more spontaneous rebooting, my logs are clean, and most importantly: I feel empowered :-)</p>
<p>I have to say I do not see Open-source firmware disappearing any time soon. For tinkerers like me who like to have complete control over their network, this is absolutely brilliant.</p>
<p>Edit: 2016-12-09 Adding some config files and diagnostics to this page, might be helpful if you are trying to replicate this</p>
<p>/etc/config/wireless defines two access points: ap24 for 2.4GHz and ap50 for 5GHz, with passwords SECRETPASSWORD.</p>
<pre class="literal-block">
config wifi-device 'radio0'
option type 'mac80211'
option hwmode '11g'
option path 'platform/qca956x_wmac'
option txpower '20'
option country 'FR'
option distance '50'
option channel '3'
config wifi-iface
option device 'radio0'
option network 'lan'
option mode 'ap'
option ssid 'ap24'
option encryption 'psk2+tkip+ccmp'
option macfilter 'deny'
option key 'SECRETPASSWORD'
config wifi-device 'radio1'
option type 'mac80211'
option hwmode '11a'
option path 'pci0000:00/0000:00:00.0'
option htmode 'VHT80'
option txpower '20'
option country 'FR'
option distance '50'
option channel '136'
config wifi-iface
option device 'radio1'
option network 'lan'
option mode 'ap'
option ssid 'ap50'
option encryption 'psk2+tkip+ccmp'
option key 'SECRETPASSWORD'
</pre>
<p>Here are the outputs of iw phy0 info:</p>
<pre class="literal-block">
#iw phy0 info
Wiphy phy0
max # scan SSIDs: 16
max scan IEs length: 199 bytes
max # sched scan SSIDs: 0
max # match sets: 0
Retry short limit: 7
Retry long limit: 4
Coverage class: 0 (up to 0m)
Device supports AP-side u-APSD.
Available Antennas: TX 0x3 RX 0x3
Configured Antennas: TX 0x3 RX 0x3
Supported interface modes:
* managed
* AP
* AP/VLAN
* monitor
* mesh point
Band 2:
Capabilities: 0x19ef
RX LDPC
HT20/HT40
SM Power Save disabled
RX HT20 SGI
RX HT40 SGI
TX STBC
RX STBC 1-stream
Max AMSDU length: 7935 bytes
DSSS/CCK HT40
Maximum RX AMPDU length 65535 bytes (exponent: 0x003)
Minimum RX AMPDU time spacing: 8 usec (0x06)
HT TX/RX MCS rate indexes supported: 0-15
VHT Capabilities (0x338001b2):
Max MPDU length: 11454
Supported Channel Width: neither 160 nor 80+80
RX LDPC
short GI (80 MHz)
TX STBC
RX antenna pattern consistency
TX antenna pattern consistency
VHT RX MCS set:
1 streams: MCS 0-9
2 streams: MCS 0-9
3 streams: not supported
4 streams: not supported
5 streams: not supported
6 streams: not supported
7 streams: not supported
8 streams: not supported
VHT RX highest supported: 0 Mbps
VHT TX MCS set:
1 streams: MCS 0-9
2 streams: MCS 0-9
3 streams: not supported
4 streams: not supported
5 streams: not supported
6 streams: not supported
7 streams: not supported
8 streams: not supported
VHT TX highest supported: 0 Mbps
Frequencies:
* 5180 MHz [36] (20.0 dBm)
* 5200 MHz [40] (20.0 dBm)
* 5220 MHz [44] (20.0 dBm)
* 5240 MHz [48] (20.0 dBm)
* 5260 MHz [52] (20.0 dBm) (radar detection)
DFS state: usable (for 2032442 sec)
DFS CAC time: 60000 ms
* 5280 MHz [56] (20.0 dBm) (radar detection)
DFS state: usable (for 2032442 sec)
DFS CAC time: 60000 ms
* 5300 MHz [60] (20.0 dBm) (radar detection)
DFS state: usable (for 2032442 sec)
DFS CAC time: 60000 ms
* 5320 MHz [64] (20.0 dBm) (radar detection)
DFS state: usable (for 2032442 sec)
DFS CAC time: 60000 ms
* 5500 MHz [100] (27.0 dBm) (radar detection)
DFS state: available (for 2032379 sec)
DFS CAC time: 60000 ms
* 5520 MHz [104] (27.0 dBm) (radar detection)
DFS state: available (for 2032379 sec)
DFS CAC time: 60000 ms
* 5540 MHz [108] (27.0 dBm) (radar detection)
DFS state: available (for 2032379 sec)
DFS CAC time: 60000 ms
* 5560 MHz [112] (27.0 dBm) (radar detection)
DFS state: available (for 2032379 sec)
DFS CAC time: 60000 ms
* 5580 MHz [116] (27.0 dBm) (radar detection)
DFS state: usable (for 2032442 sec)
DFS CAC time: 60000 ms
* 5600 MHz [120] (27.0 dBm) (radar detection)
DFS state: usable (for 2032442 sec)
DFS CAC time: 60000 ms
* 5620 MHz [124] (27.0 dBm) (radar detection)
DFS state: usable (for 2032442 sec)
DFS CAC time: 60000 ms
* 5640 MHz [128] (27.0 dBm) (radar detection)
DFS state: usable (for 2032442 sec)
DFS CAC time: 60000 ms
* 5660 MHz [132] (27.0 dBm) (radar detection)
DFS state: usable (for 2032442 sec)
DFS CAC time: 60000 ms
* 5680 MHz [136] (27.0 dBm) (radar detection)
DFS state: usable (for 2032442 sec)
DFS CAC time: 60000 ms
* 5700 MHz [140] (27.0 dBm) (radar detection)
DFS state: usable (for 2032442 sec)
DFS CAC time: 60000 ms
* 5720 MHz [144] (disabled)
* 5745 MHz [149] (disabled)
* 5765 MHz [153] (disabled)
* 5785 MHz [157] (disabled)
* 5805 MHz [161] (disabled)
* 5825 MHz [165] (disabled)
valid interface combinations:
* #{ AP, mesh point } <= 8, #{ managed } <= 1,
total <= 8, #channels <= 1, STA/AP BI must match, radar detect widths: { 20 MHz (no HT), 20 MHz, 40 MHz, 80 MHz }
HT Capability overrides
* MCS: ff ff ff ff ff ff ff ff ff ff
* maximum A-MSDU length
* supported channel width
* short GI for 40 MHz
* max A-MPDU length exponent
* min MPDU start spacing
Device supports VHT-IBSS.
</pre>
<p>Here are the ouputs of iw phy1 info:</p>
<pre class="literal-block">
# iw phy1 info
Wiphy phy1
max # scan SSIDs: 4
max scan IEs length: 2257 bytes
max # sched scan SSIDs: 0
max # match sets: 0
Retry short limit: 7
Retry long limit: 4
Coverage class: 1 (up to 450m)
Device supports AP-side u-APS.
Device supports T-DLS.
Available Antennas: TX 0x3 RX 0x3
Configured Antennas: TX 0x3 RX 0x3
Supported interface modes:
* IBSS
* managed
* AP
* AP/VLAN
* WDS
* monitor
* mesh point
* P2P-client
* P2P-GO
* outside context of a BSS
Band 1:
Capabilities: 0x11ee
HT20/HT40
SM Power Save disabled
RX HT20 SGI
RX HT40 SGI
TX STBC
RX STBC 1-stream
Max AMSDU length: 3839 bytes
DSSS/CCK HT40
Maximum RX AMPDU length 65535 bytes (exponent: 0x003)
Minimum RX AMPDU time spacing: 8 usec (0x06)
HT TX/RX MCS rate indexes supported: 0-15
Frequencies:
* 2412 MHz [1] (20.0 dBm)
* 2417 MHz [2] (20.0 dBm)
* 2422 MHz [3] (20.0 dBm)
* 2427 MHz [4] (20.0 dBm)
* 2432 MHz [5] (20.0 dBm)
* 2437 MHz [6] (20.0 dBm)
* 2442 MHz [7] (20.0 dBm)
* 2447 MHz [8] (20.0 dBm)
* 2452 MHz [9] (20.0 dBm)
* 2457 MHz [10] (20.0 dBm)
* 2462 MHz [11] (20.0 dBm)
* 2467 MHz [12] (20.0 dBm)
* 2472 MHz [13] (20.0 dBm)
* 2484 MHz [14] (disabled)
valid interface combinations:
* #{ managed } <= 2048, #{ AP, mesh point } <= 8, #{ P2P-client, P2P-GO } <= 1, #{ IBSS } <= 1,
total <= 2048, #channels <= 1, STA/AP BI must match, radar detect widths: { 20 MHz (no HT), 20 MHz, 40 MHz }
* #{ WDS } <= 2048,
total <= 2048, #channels <= 1, STA/AP BI must match
HT Capability overrides:
* MCS: ff ff ff ff ff ff ff ff ff ff
* maximum A-MSDU length
* supported channel width
* short GI for 40 MHz
* max A-MPDU length exponent
* min MPDU start spacing
</pre>
<p>Have fun with your new Access Point!</p>
easy-rsa alternative2015-12-30T22:52:00+01:002015-12-30T22:52:00+01:00nicolas314tag:maz.one,2015-12-30:/easy-rsa-alternative.html<p>Glad to announce that 2cca, the two-cent Certification Authority has now been ported to pure C with libcrypto (openssl) as single dependency. The goal was to make it available on openwrt as it seems pyopenssl is not available on this platform -- without a lot of efforts.</p>
<p>As always, I swear …</p><p>Glad to announce that 2cca, the two-cent Certification Authority has now been ported to pure C with libcrypto (openssl) as single dependency. The goal was to make it available on openwrt as it seems pyopenssl is not available on this platform -- without a lot of efforts.</p>
<p>As always, I swear this is the last time I ever link one of my sources against OpenSSL... until a replacement is made available.</p>
<p>Back to the point: you can now generate a Root CA, server, and client certificates to use with OpenVPN, with a couple of commands.</p>
<p>Download it from here:</p>
<p><a class="reference external" href="https://github.com/nicolas314/2cca">https://github.com/nicolas314/2cca</a></p>
<p>Compile it with:</p>
<pre class="literal-block">
cc -o 2cca 2cca.c -lcrypto
</pre>
<p>Generate a root with e.g.:</p>
<pre class="literal-block">
2cca root O=Home CN=MyRootCA C=FR L=Paris email=postmaster@example.com
</pre>
<p>Your root is entirely defined by ca.crt and ca.key in the current directory. Its default duration is 10 years. Now that you have a root, you are going to use it to sign server and client certificates with e.g.:</p>
<pre class="literal-block">
2cca server CN=vpn.example.com C=FR L=Roubaix email=vpnmaster@example.com
</pre>
<pre class="literal-block">
2cca client CN=jdoe C=UK L=London email=jdoe@example.com duration=365
</pre>
<p>Your server identity is defined by vpn.example.com.crt and vpn.example.com.key. Your first client is jdoe.crt/jdoe.key.</p>
<p>You can verify certificates using openssl verify, e.g.:</p>
<pre class="literal-block">
openssl verify -CAfile ca.crt jdoe.crt
</pre>
<p>Certificate serial numbers are 128-bit long, which guarantees that they can be unique without having to memorize an incremental index. Your certificate database is the current directory.</p>
<p>Enjoy!</p>
Easier easy-rsa2015-12-28T00:51:00+01:002015-12-28T00:51:00+01:00nicolas314tag:maz.one,2015-12-28:/easier-easy-rsa.html<img alt="OpenVPN" src="/images/openvpn.png" style="width: 300px; height: 300px;" />
<p>If you have ever set up an OpenVPN server, you probably had to fight your way through the certificate generation steps. Something like what is detailed here:</p>
<p><a class="reference external" href="https://openvpn.net/index.php/open-source/documentation/miscellaneous/77-rsa-key-management.html">https://openvpn.net/index.php/open-source/documentation/miscellaneous/77-rsa-key-management.html</a></p>
<p>The official OpenVPN guide refers to easy-rsa, which is a royal pain in …</p><img alt="OpenVPN" src="/images/openvpn.png" style="width: 300px; height: 300px;" />
<p>If you have ever set up an OpenVPN server, you probably had to fight your way through the certificate generation steps. Something like what is detailed here:</p>
<p><a class="reference external" href="https://openvpn.net/index.php/open-source/documentation/miscellaneous/77-rsa-key-management.html">https://openvpn.net/index.php/open-source/documentation/miscellaneous/77-rsa-key-management.html</a></p>
<p>The official OpenVPN guide refers to easy-rsa, which is a royal pain in the butt. Even with the HOWTO in front of me, it takes me ages to set things up and if I ever have to come back later to generate more client certificates, I inevitably end up restarting from scratch because I cannot remember which steps I took and where I stored files.</p>
<p>Does not seem so difficult though. You need to generate a Root CA, and then use it to sign a server certificate (which is stored on your server) and client certificates which you distribute to your clients. I re-implemented the whole thing as a Python script in a couple of hours, tested it with an openvpn instance, and it works quite well. The script can be found here:</p>
<p><a class="reference external" href="https://github.com/nicolas314/2cca">https://github.com/nicolas314/2cca</a></p>
<p>It is called <em>two-cent CA</em> because that is exactly what it is. There is no support for security modules like smart cards or HSMs because I do not need them, but since it is based on python-openssl it should not be too hard to make it work with P11 tokens.</p>
<p>Here is an example session where I create the root, a server identity, and two client identities for Alice and Bob.</p>
<pre class="literal-block">
$ python 2cca.py root
Give a name to your new root authority (default: Root CA)
Name: MyRoot
Which country is it located in? (default: ZZ)
Provide a 2-letter country code like US, FR, UK
Country: ZZ
Which city is it located in? (optional)
City:
What organization is it part of? (default: Home)
Organization: Home
--- generating key pair (2048 bits)
Specify a certificate duration in days (default: 3650)
Duration:
--- self-signing certificate
--- saving results to root.crt and root.key
done
</pre>
<pre class="literal-block">
$ python 2cca.py server
--- loading root certificate and key
Give a name to your new server (default: openvpn-server)
Name: myopenvpn-server
Which country is it located in? (default: ZZ)
Provide a 2-letter country code like US, FR, UK
Country: ZZ
Which city is it located in? (optional)
City:
--- generating key pair (2048 bits)
Specify a certificate duration in days (default: 3650)
Duration:
--- signing certificate with root
--- saving results to myopenvpn-server.crt and myopenvpn-server.key
</pre>
<pre class="literal-block">
$ python 2cca.py client
--- loading root certificate and key
Give a name to your new client (default: openvpn-client)
Name: Alice
Which country is it located in? (default: ZZ)
Provide a 2-letter country code like US, FR, UK
Country: UK
Which city is it located in? (optional)
City: Cambridge
--- generating key pair (2048 bits)
Specify a certificate duration in days (default: 3650)
Duration:
--- signing certificate with root
--- saving results to Alice.crt and Alice.key
</pre>
<pre class="literal-block">
$ python 2cca.py client
--- loading root certificate and key
Give a name to your new client (default: openvpn-client)
Name: Bob
Which country is it located in? (default: ZZ)
Provide a 2-letter country code like US, FR, UK
Country: US
Which city is it located in? (optional)
City: Boston
--- generating key pair (2048 bits)
Specify a certificate duration in days (default: 3650)
Duration:
--- signing certificate with root
--- saving results to Bob.crt and Bob.key
</pre>
<pre class="literal-block">
& ls
2cca.py Alice.key Bob.key myopenvpn-server.crt root.crt
Alice.crt Bob.crt README.md myopenvpn-server.key root.key
</pre>
<p>You want to keep root.crt for what OpenVPN calls the CA certificate. Do not loose root.key, you will need it whenever you will want to issue more client or server certificates. Install the other files as required.</p>
<p>Tested on Linux (Debian, Archlinux) and OSX.</p>
<p>Enjoy!</p>
OpenWRT on MR30202015-12-09T00:31:00+01:002015-12-09T00:31:00+01:00nicolas314tag:maz.one,2015-12-09:/openwrt-on-mr3020.html<img alt="TP-Link MR3020" src="/images/mr3020.jpg" style="width: 300px; height: 300px;" />
<p>Situation: you live in a one-room appartment and receive Internet through a single RJ45 plug in the wall. How do you extend it to all devices in the room for the cheapest price?</p>
<p>Objective: provide WiFi connectivity on a dedicated LAN where users can see each other and share files …</p><img alt="TP-Link MR3020" src="/images/mr3020.jpg" style="width: 300px; height: 300px;" />
<p>Situation: you live in a one-room appartment and receive Internet through a single RJ45 plug in the wall. How do you extend it to all devices in the room for the cheapest price?</p>
<p>Objective: provide WiFi connectivity on a dedicated LAN where users can see each other and share files easily. The system should be easy to repair or upgrade. Bonus points for extra functionalities like VPN provided to the whole WiFi LAN, ad filtering, or shared folders on Samba.</p>
<p>My first go-to for anything cheap related to computing is the RaspberryPi. How can you beat it? For 35 euros (amazon.fr in December 2015) you get a full-powered Linux box with RJ45 and enough USB ports to power an external disk and a WiFi dongle. I gave it a try and came to the conclusion that the result would probably be a little too expensive and probably brittle. I have several WiFI USB adapters lying around and none of them was able to create a WiFi Access Point on a RPi, though they do work flawlessly as WiFi clients. They almost got there but not completely. You need to recompile your own WiFi access point software, and taking care of my own iptables rules is just beyond my patience.</p>
<p>Better option: Openwrt! This open-source Linux distribution is not meant for your average PC but to run on screen-less network appliances. You won't find desktop apps or anything related to X11 but you have all possible network daemons and tools running there. Openwrt runs on low-power processors like ARM, MIPS, and x86 too of course. It comes bundled with its own package distribution tools (opkg) making administration relatively easy. It is really meant for tinkerers, people who like to open the box and modify what's inside to do different stuff.</p>
<p>Obvious applications for Openwrt are of course network-related. Build your home router for 25 euros supporting IPv6, customized firewalling, guest WiFi, kid protection, quality of service, network monitoring, Virtual Private Networks, or file sharing, to name a few. A popular application is PirateBox: start the box, create a local WiFi network, all clients connected to it can easily share files through the local access point. Other cool projects include running your own home telephony over the Internet (asterisk), making a sound box or an Internet radio.</p>
<p>There are also thousands of cool projects if you are so inclined and ready to take your soldering iron out of the dust. Most router hardware parts have hackable GPIO ports you can connect to a breadboard to pilot a 0-5V trigger or even read signals at a reasonable frequency. Check out the DIY section at the bottom of this page to get a few ideas:</p>
<p><a class="reference external" href="https://wiki.openwrt.org/toh/tp-link/tl-wr703n">https://wiki.openwrt.org/toh/tp-link/tl-wr703n</a></p>
<p>Note that the WR703N is a 10-20 euro piece of hardware so it is about half of the retail price of an Arduino with Ethernet shield, or a RaspberryPi. Sure, you get less hardware, but if your project is purely network-oriented this is by far a better alternative.</p>
<p>Back to our task: find cheap hardware that gets Internet from an RJ45 plug and offers decent WiFi for a few devices. I set my limit to 25 euros and chose the TP-Link MR3020. It has the minimal 4MB of internal storage needed to install Openwrt. You read that right: 4 megabytes of persistent storage, in an era where your average smartphone sports several gigabytes of RAM.</p>
<p>There are several ways to obtain an Openwrt image for a given piece of hardware. First and simplest one is to download it directly off openwrt.org. That does not always fit the bill though, because the team had to make choices about the packages that are present in each image. If space is tight (4 megs is kind of narrow) you may not be able to install anything more than was pre-bundled already. That is actually the case for the MR3020. Solution: extend the local storage. So I plugged in an 8GB USB dongle (5 euros).</p>
<p>Plug the USB in, wait, and... nothing. 'dmesg' shows the kernel recognized something USB but did not offer to mount the filesystem. Of course: the kernel modules have been trimmed down to a minimum and USB storage was not in. Quick install with opkg? No way. There is not enough space left to install a mere kernel module for USB storage so the default image falls a bit flat. Removing packages also does not work. I first tried to trim down all un-needed packages until I realized that every time I uninstalled something, space was running even lower on the root filesystem. How comes?</p>
<p>To understand why, you need to know that the default firmware image is built on top of squashfs, a read-only compressed filesystem. On top of that, openwrt adds an overlay filesystem, an image that registers all changes from the underlying read-only partition. When you delete a file from such a setup you actually write on top of it in the overlay filesystem a mention that this file should not appear any more. There is another option using a read-write filesystem (jffs2) for the firmware but apparently it can get into trouble on many routers so I did not even try.</p>
<p>One solution is to build your own openwrt image and embed only the modules you need. That one is tough. You need to dedicate an x64 machine for that, download a million source files, prepare all the compilers and build tools you ever heard of, fine tune the package configuration, press make, and wait for a few hours. You end up with an image a bit smaller than the four dreaded megabytes with a build system taking about 3GB of space on the build host.</p>
<p>To be fair, the openwrt build system is a wonderful thing. It starts by creating its own toolchain for cross-compilation for your target hardware, then builds up a whole Unix from scratch and ends up packaging everything into this tiny image. Very impressive. I have seen very few professional build systems that are as clean as this one. Congratulations to the Openwrt for working out such a beautiful system!</p>
<p>If you are not into compiling your own stuff, you have another option:</p>
<p><a class="reference external" href="https://wiki.openwrt.org/doc/howto/obtain.firmware.generate">Image Generator</a></p>
<p>This method uses pre-built components which you assemble into a minimal firmware image. That goes a lot faster (a matter of seconds) and does not eat up gigabytes of disk space. Select the few minimal modules you need to at least boot up the router on the network, and be able to mount USB storage. That is what I ended up doing.</p>
<p>I flashed an image that contained just enough software to run a kernel with network support (duh), USB storage support, package management, and some basic utilities. Plugging the USB storage worked immediately. Yay!</p>
<p>Let's start again: I have 8GB of space on USB dongle, far more than I will ever need for the router itself. I created a first ext4 partition for 700 megs, which should be enough to hold every possible package, configuration files, and even log files if needed. I added a second 128 meg partition for swap space, and left the rest as a single ext4 partition.</p>
<p>The router was booted again with USB dongle plugged in. Let it boot, log onto it as root, and set up the root filesystem as an overlay from the USB first partition. Reboot the router and there you go: your router space has grown from 4 megs to 700.</p>
<p><a class="reference external" href="https://wiki.openwrt.org/doc/howto/extroot">This brilliant howto</a>
explains exactly how to do it:</p>
<p>Once you have enough space you can start adding back all the packages you need. Just to give you an idea, I have there:</p>
<ul class="simple">
<li>A web interface to set everything up</li>
<li>Tools to monitor the device and display nice graphs on the web interface</li>
<li>DNS server, WiFi Access Point, Firewall</li>
<li>Development tools: gcc, git, python, lua, strace, screen, vim</li>
<li>A Samba server</li>
<li>sshfs to mount remote filesystems locally</li>
<li>OpenVPN either as client or server</li>
</ul>
<p>After that it is a matter of taking time to configure each service specifically. The Openwrt wiki pages are a real treasure in this respect. Everything you need is there, together with additional ideas for cool stuff to do. One hour later I was done adding VPN support, Samba service for the remaining of the USB dongle space, an ad blocker, a local web site, and a local fixed address for administration (an alias on eth0).</p>
<p>There is one point really worth mentioning about the Openwrt philosophy. If you have some Unix experience, you know that every piece of server software you install comes with its own system of configuration directories and files with a specific new syntax to learn, a different place to find log files, and another way to handle PID files. Sure there is some standardization on the way with most config files under /etc and logs under /var/log, but it seems everybody needs to invent a new syntax for configuration and logging.</p>
<p>Openwrt places all configuration files in a single place and they all respect the same key/value syntax. It is straightforward to understand, edit, diff, store in version control, or pretty-print. That alone helps smooth out the learning curve every step of the way. Gives you a definite feeling of things done in a proper, consistent, and very clean manner. So there you go: for all your network-y things, you know where to look. The Internet of things starts right here with these boxes, guys.</p>
<p>My next endeavours with OpenWRT will probably need my soldering iron. Those GPIOs on the board look tempting...</p>
Meet the Guru2015-07-07T23:50:00+02:002015-07-07T23:50:00+02:00nicolas314tag:maz.one,2015-07-07:/meet-the-guru.html<p><img alt="Math formulae" src="/images/math-formulae.png" style="width: 300px; height: 162px;" /></p>
<p>Tonight I met a guru. Not an expert, a real guru: the kind of spiritual
guide who has theories, speaks wonders, and has followers. This all started
when a friend of mine invited me over to have a quick chat with this
phenomenal genius from Canada who has invented methods …</p><p><img alt="Math formulae" src="/images/math-formulae.png" style="width: 300px; height: 162px;" /></p>
<p>Tonight I met a guru. Not an expert, a real guru: the kind of spiritual
guide who has theories, speaks wonders, and has followers. This all started
when a friend of mine invited me over to have a quick chat with this
phenomenal genius from Canada who has invented methods that are about to
bring a complete revolution to the way we deal with digital data. Data
compression to insane ratios, database speedups, multi-dimensional fractal
volumes held in a single SMS, you name it. My friend asked for advice about
the guy's theories because he wants to invest money and create a startup.</p>
<p>I was a bit dubious at first, but hey... curiosity took over. I absolutely
had to know what the guy was about. We met around a few beers and I asked
casually: "so what is it you are doing, exactly?"</p>
<p>Without hesitation, he said: "We have found methods to encode information like tight N-spaces in base 8 around bijective radical algorithms that inject coordinates inside an index that compresses the whole universe in a single integer that can be used to decompose irrational numbers like inverted matrices. You see? Using just one number, N-dimensional spaces can be represented along a bijective access ramp thanks to multi-variate polynomials."</p>
<div class="line-block">
<div class="line">"Oh wow. Continue?"</div>
<div class="line">He grinned, convinced he tossed enough big words in one sentence to lose me completely. The two disciples around him gently nodded as he went along. He stared right into my eyes as he spoke, never looking for his words, not showing an instant of hesitation.</div>
</div>
<p>"According to my method, we re-arrange coefficients in matrices in the most efficient way, heading straight to the core of multi-dimentional data spread over a lattice of chosen binomial functions, up to the power 1,000 or more. These functions are space-filling, they can visit every point in any volume exactly once."</p>
<p>A pause. I asked: "Interesting. So what are the applications?"</p>
<p>He jumped on his seat: "The whole digital world can now be expressed in real-time!" Still calm, I asked: "give me an example?"</p>
<p>"When you need to identify that a point belongs to a region in a multi-dimensional space, you only need to compute its index along a multi-variate polynomial algorithm that expands the number irregularity into a local singularity to spread out a potential zero."</p>
<p>Both disciples were still nodding. I was starting to feel uneasy. Among the waves of technical, unrelated words, I could feel he was trying to get somewhere. He was trying to encode data into another representation that has magic powers, but the mysterious polynomial functions were hard to grasp, and the application of this encoding still unclear. I ventured:</p>
<p>"How does this compare to a Fourier transform, for example?"</p>
<p>That one caught him off-balance. Apparently he did not master the vocabulary associated to that field yet, so he just waved the idea away.</p>
<p>"Fourier is bullshit. I am talking about Gödelization of the digital world. You know Gödel, right? He was a genius who encoded the world into numbers!"</p>
<p>"Yes he was, but he only did that to prove a point about reflectivity, not as an efficient encoding. Gödel numbers are only a tool to prove his point. Never heard of the term Gödelisation though. Did you invent it?"</p>
<p>"Nonsense!" he yelled. "Gödelization is helping us create an index that gives you the whole information about anything you want in just one single number!"</p>
<p>"Oh right. So you compress information then?"</p>
<p>"No, no, no". He looks worried now. "Nothing about compression."</p>
<p>"But you said earlier..."</p>
<p>"Don't interrupt me. This is not about compression, this is about efficient algorithms to manage N-dimensional multivariate volumes in NP-space by running in Newtonian arithmetic series as they expand into infinity."</p>
<p>That went on for another few minutes. As requested, I did not interrupt. When he seemed finished tossing more words at me, I asked again:</p>
<p>"So what is it you do, exactly?"</p>
<p>At that point he got the message that he was not getting me. His two disciples started moving uncomfortably on their chairs. He talked a bit louder, nailing me with his eyes as he declared his principles like bible verses, dumping more pseudo-scientific sentences into the pot like water in a well.</p>
<div class="line-block">
<div class="line">At some point he said:</div>
<div class="line">"I am working with the greatest worldwide specialists on that topic! Scientists at the Paris Observatory are using my computations for astronomy!"</div>
</div>
<p>There we go. Argument by authority: there are people more intelligent than you who believe me, so you have to believe me.</p>
<p>"That right? As it happens I worked with the Paris observatories for a decade. Who do you work with? Is that in Paris 14 or Meudon?"</p>
<p>Blank stare. Ouch, he did not see that one coming. "I work with... Jean-Pierre uh... Letruc."</p>
<p>"Letruc?"</p>
<p>"Yes, he is a world famous scientist who computes singularities."</p>
<p>I almost picked up my smartphone to look the name up but common sense prevailed. I think he would have slapped me for doubting his word.</p>
<p>"Never heard about Letruc but who cares. What do they use it for, in astronomy?"</p>
<p>"Well, you know, don't you? Computing galaxies and stars and stuff."</p>
<p>"Computing what?"</p>
<p>He sighed, as if it was obvious. "Computing the movements of multi-dimensional Euclidian forms in the space-time continuum." Duh.</p>
<p>Another pause for a few seconds. That last one was worth gold. And then he started again for the next ten minutes, bashing every possible kind of notion at me as if hitting me on the head with four-syllable words was meant to convince me. He dumped everything he had: information theory and Shannon's theorem, Euclides spaces, prime numbers, Newton laws, Fibonacci rules, and Gödel's theorem, again.</p>
<p>I said: "I am really dumb so explain me again: what can I use your... algorithms for?"</p>
<p>He jumped on that one: "Imagine you have to sort out character strings in lexicographic order. So A, then B, then C, etc. If you take a whole dictionary it might take years to achieve! There are no algorithms to achieve that efficiently."</p>
<p>"Beg your pardon, there are quite a few. Lesson 1 in any computer science course. I think you and I read the same books."</p>
<p>"Don't interrupt me! I have read 25,000 books and I have them all at home! You academics think you know everything right? You need to think differently! My mathematics have no relationship whatsoever with anything you might have learned and you are too limited to understand the implication of what I am telling you!"</p>
<p>"Sure. Sorry. Please explain again?"</p>
<p>"So you take the strings in input and you choose a multi-variate polynomial function of rank N that maps digital inputs into a real-time computation to yield a single integer number. Then you sort the list of numbers and you are done. In real-time. And even Mr Oracle out there with his powerful databases cannot do anything about that. Hah!"</p>
<div class="line-block">
<div class="line">Ah... Now we were getting somewhere. You transform a list of character strings into numbers to sort them.</div>
<div class="line">I asked: "This... Gödelization, to use your term, has a cost, no?"</div>
</div>
<p>"No! It is instantaneous! Real-time! The greatest scientist with super-computers can never find a better transformation because the polynomials are..."</p>
<p>"Ok. It has no cost. And then you are sorting numbers."</p>
<p>"Don't interrupt me! You need to listen! You guys never listen, but you might learn something". He was now literally barking at me so I asked him to calm down several times, without result.</p>
<p>He started again: "I am working with the greatest minds on this planet on this topic! The guys at INRIA admire my work!"</p>
<p>"So you work with INRIA? You know I did, too? Which office? Nice? Paris?"</p>
<p>That one hit him like a rock. He stuttered: "I... I... I... worked in Paris and gave a talk in Sophia Antipolis a few months ago. But you could not understand anything I said there, you would have to read a very long paper I wrote, 120 pages, which explains everything."</p>
<p>"Please share the link, I am impatient to read it."</p>
<p>"You would not understand anyway! People oppose me on the principles that I am trying to do things differently but they are WRONG! They are all WRONG!"</p>
<p>He got so infuriated at that point that one of his disciples left the room while the other decided to take over and explain to me the principles of integer sorting, as if he was talking to a five year old. I tried to interact but was systematically asked to shut up and listen to an endless stream of made-up scientific words, picking notions from mathematics, physics, electronics. The guy stressed every one of his points by quoting famous scientists and laws that made absolutely no sense, like Newton's law of primes, the well-known Euclides and Peano numbers, or the famous Fibonacci theorem.</p>
<p>"So you take data and transform it into numbers?"</p>
<p>"NOOO!!! YOU DON'T GET IT, DO YOU??"</p>
<p>"Apparently I don't. But you just said that you map strings into numbers."</p>
<p>"NO! You don't understand anything. Listen to me..." And then five more minutes of vocal diarrhea meant to shut me up.</p>
<div class="line-block">
<div class="line">After a while I asked:</div>
<div class="line">"Is there any point in me asking questions? Or do you just want to listen to yourself talking?"</div>
</div>
<p>That was pure provocation, I admit, but getting yelled at and being told I was too stupid to understand his genius did not help much in making me comfortable. I left the room as he was erupting in anger. His disciple yelled at me while I was going out: "Ha! You do not even understand the concept of bijection, you dumb ass!"</p>
<p>I could not have taken a single more second of that bullshit. Took me some time to calm down and then I tried to reflect on what I had just witnessed. What is this guy trying to achieve exactly? What is his agenda?</p>
<p>I saw a guy who has mastered a talent to throw scientific-looking words and concepts into sentences, around an almost-believable story, ending up with promises of being rich by selling never-done-before powers over digital data. The fact that he has disciples and tries to recruit more is extremely interesting, though I still cannot figure out what he is gaining there beyond recognition by a few gullible souls. With such a character, latent paranoia is expected to be found: people do not understand him, they would not listen, he is fighting against the establishment, and most probably: other scientists and engineers are conspiring against him because they know he has a truth everybody was dreaming of and nobody could find or understand.</p>
<p>Interesting character. In my life I met quite a few other gurus: a medical shaman who is the only person in the world who knows about the true powers of plants, an artist turned into a religious leader of his own sect who teaches philosophy, psychoanalysis, and black magic in the same session. There are also the conspiracy guys who know every secret about the NSA, the FBI, the CIA, why Americans never walked on the Moon or destroyed the Twin Towers themselves. These guys scare me to no end. After spending five minutes with them you realize they do not see you or anybody around them for that matter. Other people are just witnesses of the fact that they have some untold truth only known to them, which everybody wants, and nobody would ever believe.</p>
<p>The good thing with having so many voices in your head is that you never feel alone.</p>
<p>Relevant XKCD: <a class="reference external" href="https://xkcd.com/451/">XKCD 451</a></p>
XML just called2015-01-27T23:59:00+01:002015-01-27T23:59:00+01:00nicolas314tag:maz.one,2015-01-27:/xml-just-called.html<img alt="Mean science (cartoon)" src="/images/mean-science.jpg" style="width: 350px; height: 371px;" />
<p>Lunch time. I had been waiting outside for my colleagues to show up for close to 10 minutes when they said they were following me. Getting impatient, I wrote an SMS to my office mate containing the single word: <strong>MANGER.</strong> Interestingly enough, Android does not allow me to set a …</p><img alt="Mean science (cartoon)" src="/images/mean-science.jpg" style="width: 350px; height: 371px;" />
<p>Lunch time. I had been waiting outside for my colleagues to show up for close to 10 minutes when they said they were following me. Getting impatient, I wrote an SMS to my office mate containing the single word: <strong>MANGER.</strong> Interestingly enough, Android does not allow me to set a default phone number for a contact, and the SMS was sent to a land line instead of my colleague's mobile.</p>
<p>The guys showed up, we had lunch, and came back to the office. My colleague saw the red light blinking on his land line and pressed the loudspeaker button without much thinking. We were served a bit of classical music, and then a synthetic voice declared without any emotion: <em>"You have received a message. Reading: OPEN BRACKET EMPHASIS CLOSE BRACKET MANGER OPEN BRACKET SLASH EMPHASIS CLOSE BRACKET."</em></p>
<p>Best laugh we've had in a while.</p>
<p>My next challenge: get the phone system to read me a whole stack trace.</p>
Je suis Charlie2015-01-14T00:44:00+01:002015-01-14T00:44:00+01:00nicolas314tag:maz.one,2015-01-14:/je-suis-charlie.html<p>The Paris events were dreadful. Being assassinated for drawing cartoons is
probably the most stupid thing that could ever happen. I am particularly
touched by the death of Cabu, a cartoonist who, to me, has been part of the
French culture forever. Whenever something of importance happened
worldwide, Cabu was …</p><p>The Paris events were dreadful. Being assassinated for drawing cartoons is
probably the most stupid thing that could ever happen. I am particularly
touched by the death of Cabu, a cartoonist who, to me, has been part of the
French culture forever. Whenever something of importance happened
worldwide, Cabu was always there to draw it with a short, to-the-point
cartoon that would highlight absurdity in the smartest and most hilarious
way. When I was a kid Cabu was drawing stuff live in kid TV shows. It
really feels like losing a soul brother who got killed for drawing smart
stuff. Charlie Hebdo is purely about provocation and pushing freedom of
speech to its limits, and Cabu was a genius for defusing tense situations
by making everybody laugh. His cartoons went beyond just laughter, he
really made you think twice about the topics at hand.</p>
<p>The aftermath was also dreadful. The manhunt, more killings, the hostage
situation inside Paris. Every piece of news was worse than the previous
one, until the police put an end to it.</p>
<p>Anyway, I did not go to the march. I would have wanted to but taking my
kids to such a crowd would have been difficult to say the least. There are
other ways to participate, like having a 2-hour conversation with my boys
about what happened, why freedom of speech is capital to preserve our way
of life, who these cartoonists were, and how some very stupid people can be
manipulated in creating chaos, bringing the worst to themselves in the
doing. When you read the third guy's conversations with his hostages and
hear what he left in the video that was published after his death, you
realize that you are not facing an extremist but somebody who probably has
trouble making a complete intelligible sentence, let alone have a coherent
train of thought. There will always be simpletons like him, but those who
received military training should probably be monitored to avoid such
things from ever happening again.</p>
<p>I am not scared. I believe nobody is scared in France of terrorist attacks.
The most global reaction is sadness, and people appalled by such immense
stupidity.</p>
<p>If I had to keep anything positive from these events, it is the unanimous
worldwide reaction of horror that shook the whole world. Seeing all these
marches around the planet made me feel all fuzzy inside. The world is a
better place today than when I was born, and this is becoming visible.
Feels like the whole planet has grown up. There are still some dark places
left, showing we need more education than weapons. Hopefully that will
happen within our life time.</p>
Parental control with OpenWRT and OpenDNS2014-12-10T22:24:00+01:002014-12-10T22:24:00+01:00nicolas314tag:maz.one,2014-12-10:/parental-control-with-openwrt-and-opendns.html<img alt="OpenWRT prompt" src="/images/openwrt-prompt.png" style="width: 300px; height: 93px;" />
<p>The following recipe took me a whole evening to find, so I am documenting it here in hope it could be useful to somebody else.</p>
<p>I recently upgraded my home network to a beefier TP-Link C5 Archer. This little box packs two Wi-Fi access points in 2.4 and 5GHz …</p><img alt="OpenWRT prompt" src="/images/openwrt-prompt.png" style="width: 300px; height: 93px;" />
<p>The following recipe took me a whole evening to find, so I am documenting it here in hope it could be useful to somebody else.</p>
<p>I recently upgraded my home network to a beefier TP-Link C5 Archer. This little box packs two Wi-Fi access points in 2.4 and 5GHz (Wi-Fi ac), which pushes wireless speeds up to 500Mbit/s within a few meters range. The main selling point for me was that it runs the latest OpenWRT firmware with absolutely no issue whatsoever. Flash firmware, done.</p>
<p>OpenWRT has become a real Linux distribution today, packing more power than you could ever imagine achieving with such hardware. I certainly miss the Tomato user-friendly GUI, but I do enjoy the power at my fingertips when it comes to network configuration. Kudos to the OpenWRT team for such a technical achievement!</p>
<p>Back to the point: parental control. I have kids at home and all sorts of
networked devices: smartphones, tablets, computers, servers, printers, you
name it. I want to be able to disable adult site browsing and the like from
kids hardware. The easiest solution I found so far is
<a class="reference external" href="http://www.opendns.com">OpenDNS</a>, which offers you free DNS filtering
for one IP address. Create an account, configure your home IP address, set
the categories you want to ban, and done. Any machine on my internal
network using OpenDNS will receive re-directs for unwanted sites. In the
past I used to manually modify the DNS settings on all kids hardware to
switch to OpenDNS servers, but that quickly becomes old, and sometimes
requires some sleight-of-hand to configure. Forget it.</p>
<p>Enter OpenWRT: you can actually assign different DHCP settings to hosts on
your network, e.g. different DNS servers. Even if the documentation is
respectfully thick on that topic, it took me a while to understand it.</p>
<p>In its latest incarnation Barrier Breaker (Dec 2014), OpenWRT packs all DHCP information into /etc/config/dhcp. Make your modifications there and restart the dnsmasq daemon to activate them.</p>
<p>Procedure:</p>
<ol class="arabic simple">
<li>edit <cite>/etc/config/dhcp</cite> to add a new section</li>
</ol>
<pre class="literal-block">
config tag 'kids'
list dhcp_option '6,208.67.222.222,208.67.220.220'
</pre>
<ol class="arabic simple" start="2">
<li>Now add individual sections for all devices you want to include in the 'kids' section:</li>
</ol>
<pre class="literal-block">
config host
option name 'pluto'
option mac 'YOUR DEVICE MAC ADDRESS'
option ip 'YOUR DEVICE ADDRESS ON THE INTERNAL NETWORK'
option tag 'kids'
</pre>
<p>3. Restart dnsmasq with:</p>
<pre class="literal-block">
/etc/init.d/dnsmasq restart
</pre>
<p>And you are done. Just tag the hosts you want to be part of the kids zone to distribute the OpenDNS servers instead of the default one.</p>
<p>Same procedure if you want to achieve the reverse effect: set the default
DNS to be kid-friendly, tag machines that don't need to be, and assign them
another regular DNS.</p>
<p>References:</p>
<ul class="simple">
<li><a class="reference external" href="https://wiki.openwrt.org/doc/uci/dhcp">OpenWRT: Classifying Clients And Assigning Individual Options</a></li>
<li><a class="reference external" href="https://forum.openwrt.org/viewtopic.php?id=45701">Send different DNS servers to different hosts</a></li>
</ul>
PC-to-PC 1012014-10-15T00:05:00+02:002014-10-15T00:05:00+02:00nicolas314tag:maz.one,2014-10-15:/pc-to-pc-101.html<img alt="RJ-45" src="/images/rj45.jpg" style="width: 300px; height: 276px;" />
<p>Got my hands on an old but faithful laptop recently. The thing is a lot faster than my son's desktop, so I thought it would make a nice update for him. First thing I did was upgrade the laptop's disk to an SSD and install Linux Mint. This is now …</p><img alt="RJ-45" src="/images/rj45.jpg" style="width: 300px; height: 276px;" />
<p>Got my hands on an old but faithful laptop recently. The thing is a lot faster than my son's desktop, so I thought it would make a nice update for him. First thing I did was upgrade the laptop's disk to an SSD and install Linux Mint. This is now the fastest computer in the house, with a total Linux boot time averaging around 3-4 seconds from power button press to full windowed environment. Yay!</p>
<p>Next task: transfer his home directory from his desktop to his new laptop. Both machines are currently connected to the home network through Wi-Fi, and we are talking about transferring about 100 Gbytes. Even with a beefy router averaging around 1Mbyte/sec, we are talking about 24+ hours of transfer. Not ideal. One solution would be to connect an external hard drive, copy all contents, then restore on the other machine, but I thought there must be a better solution. Both machines have RJ-45 Gbit connectors, so why not take advantage of this?</p>
<p>Now comes the problem: how do you create an ad-hoc network between two Linux machines to allow them to exchange data through a single Ethernet cable? Bonus points if you do not use a router.</p>
<p>First thing that came to my mind was: install all necessary software to transform one of the PCs into a simple router. A simple DHCP server and very basic routes should be enough. But no, that is not even necessary: switch both PCs to fixed IP addresses like 10.0.0.1 and 10.0.0.2 and it should just work. Which I tried, without much success. A ping would work for a few seconds but both PCs lost their network connection almost immediately. Could not figure out why. I tried GUI configuration, modifying <em>/etc/network/interfaces</em>, manually switching with <em>ip add</em>, but nothing worked. The first packets would work fine, I could even connect with ssh, and after 10 seconds connectivity collapsed on both sides. We have been living with DHCP-configured networks for so long now that the default Linux networking tools are all assuming some kind of dynamic addressing scheme handled by a proper router. No luck.</p>
<p>I must have launched <em>ifconfig</em> half a million times and never got a proper answer about why this damn thing would disconnect itself after 30 seconds. And then suddenly I realized I had the solution to my problem just in front of me:</p>
<pre class="literal-block">
% ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::210:99ff:fe26:6db1 prefixlen 64 scopeid 0x20<link>
</pre>
<p>The answer is on the bottom line: the IPv6 link-local address, of course!</p>
<p>IPv6 local links are designed precisely to work without a router or DHCP server. Plug a cable, talk. No need for any kind of configuration, gateway, or network mask. Let's try:</p>
<pre class="literal-block">
% ping6 fe80::210:99ff:fe26:6db1%eth0
PING fe80::210:99ff:fe26:6db1%eth0(fe80::210:75ff:fe26:6db1) 56 data bytes
64 bytes from fe80::210:99ff:fe26:6db1: icmp_seq=1 ttl=64 time=0.093 ms
64 bytes from fe80::210:99ff:fe26:6db1: icmp_seq=2 ttl=64 time=0.073 ms
</pre>
<p>NB: If you use link-local IPv6 addresses, you need to suffix the address by the NIC name (%eth0). Once that was cleared, I could start an ssh session from one box to the other without more thoughts. No software installation or configuration of any kind are required.</p>
<p>The remaining bit was to find out how to use IPv6 addresses with scp. A bit
of googling revealed that scp needs IPv6 addresses to be surrounded by
escaped brackets, which produces this kind of user-friendly command-line:</p>
<pre class="literal-block">
scp -6 nicolas@\[fe80::210:99ff:fe26:6db1%eth0\]:/home/music .
</pre>
<p>With both machines running on SSD drives and on Gbit NICs, the whole 100 Gbytes took just a few minutes to transfer.</p>
<p>Today I learned IPv6 can be useful immediately.</p>
No news today2014-07-29T13:23:00+02:002014-07-29T13:23:00+02:00nicolas314tag:maz.one,2014-07-29:/no-news-today.html<p>Sarkozy was president of France during five years from 2007 to 2012. During those five long years, the French media switched to full-Sarkozy mode, as if he was the only newsworthy person on this planet. Most newspapers would not hesitate to put his picture on the front page on a …</p><p>Sarkozy was president of France during five years from 2007 to 2012. During those five long years, the French media switched to full-Sarkozy mode, as if he was the only newsworthy person on this planet. Most newspapers would not hesitate to put his picture on the front page on a daily or weekly basis, news web sites were just ablaze with news reports about what he did, what he had done, what he might have done, what people thought about what he had done or should do, together with editorials, discussions, and debates, all centered around just one single person.</p>
<p>Listening to the radio soon became a chore. The journalists were so overwhelmed with this character that everything had to be brought back to Sarkozy sooner or later. At first it was fun: counting the average number of times his name was mentioned on air brought an average frequency of one every 10 seconds or so. It quickly became old, though. After a few weeks I decided to switch off the radio as soon as his name came up. Survival air time went from one minute to: switch on radio, hear "Sarkozy", switch off radio. He literally neutralized journalism in the country for five years.</p>
<p>Since about that time, I stopped listening to French radios and switched over completely to two other news sources: BBC Radio 4, and Bayern 2. Both have comprehensive programs about all kinds of topics and news about the rest of the world and not just one single person.</p>
<p>The revelation came to me in April 2013 after reading this article:</p>
<p><a class="reference external" href="http://www.theguardian.com/media/2013/apr/12/news-is-bad-rolf-dobelli">http://www.theguardian.com/media/2013/apr/12/news-is-bad-rolf-dobelli</a></p>
<p>I realized that most news reports are centered on topics I, at best, only vaguely care about. There is nothing I can do about a plane crash, economy fluctuations, or yet another G20 meeting. I am not trying to reduce the importance of these events, just stressing that there are a lot more pressing topics I need to know about and they are not reported in the news.</p>
<p>Since about a year I stopped reading papers or listening to the radio. No TV at home for the past 20+ years, and I can finally declare myself free of these dreadful news reports every morning. I get today's weather by googling on a tablet and can now dedicate my time to reading sources I really care about.</p>
<p>Every now and then, I give it another try: switching on Radio 4 and Bayern 2 is still a joy, you just have to avoid full hours to dodge news reports. I gave up buying newspapers and removed all bookmarks to news web sites from my browsers. I still get some news from friends and colleagues and occasionally spend some time on a news topic of interest, but overall the net result is quite good. No more stress-inducing news about dreadful things out of my control, no more Sarkozyfication, no more reports of the daily death toll.</p>
<p>Net results: less sources of negativity in my life, less reasons to stress. I have so far never encountered a situation where I would think: "Gee, I wish I had known about that before, I should have listened to the news more carefully."</p>
<p>No news is good news.</p>
Planes taking off2014-03-20T01:14:00+01:002014-03-20T01:14:00+01:00nicolas314tag:maz.one,2014-03-20:/planes-taking-off.html<img alt="Composite image of planes taking off" src="/images/composite-airplanes-taking-off.jpg" />
<p>Planes taking off in Hannover. Nothing new, but I like this picture.</p>
<p>See also <a class="reference external" href="http://twistedsifter.com/2012/02/picture-of-the-day-striking-multiple-exposure-shot-of-takeoffs-at-hannover-aiport/">http://twistedsifter.com/2012/02/picture-of-the-day-striking-multiple-exposure-shot-of-takeoffs-at-hannover-aiport/</a></p>
Insanely Large Machines2013-11-23T01:10:00+01:002013-11-23T01:10:00+01:00nicolas314tag:maz.one,2013-11-23:/insanely-large-machines.html<p>Just saw this xkcd being pointed to by <a class="reference external" href="http://www.eso.org">ESO</a> today: <a class="reference external" href="https://xkcd.com/1294/">xkcd Telescope names.</a></p>
<p>The Very Large Telescope is a project I was lucky enough to help bring from
its infancy to daily operations. In the crowd of engineers who worked on
that project, Daniel Enard was considered by most as …</p><p>Just saw this xkcd being pointed to by <a class="reference external" href="http://www.eso.org">ESO</a> today: <a class="reference external" href="https://xkcd.com/1294/">xkcd Telescope names.</a></p>
<p>The Very Large Telescope is a project I was lucky enough to help bring from
its infancy to daily operations. In the crowd of engineers who worked on
that project, Daniel Enard was considered by most as the spiritual (and
technical) father of the VLT. I remember him mentioning once at lunch time
that he picked the name <em>Very Large Telescope</em> as a temporary placeholder
and it finally stuck for lack of finding a better one. Engineers may be
creative but not so good at marketing, it seems. We heard a bazillion jokes
about astronomers having something to compensate and <em>Very Large</em> finally
ceased being ridiculous as we heard it again and again. Everybody worked on
the VLT and that was it.</p>
<p>When the next project had to be named, the term <em>Extremely Large Telescope</em>
was naturally coined in reference to being bigger than <em>very large</em>. After
all the jokes we had about <em>Very Large</em> it only seemed fitting that we
would start again on <em>Extremely Large</em>. Half a bazillion jokes later,
everybody talked about ELT and we all forgot about the biggerness, again.</p>
<p>When the next project started emerging from a few busy brains with too much
time on their hands, it was obvious they had been looking to extend the
pattern towards <em>biggerness</em>. The name <em>Overwhelmingly Large</em> was probably
coined one afternoon in a Biergarten after one too many beers by a team of
French and Italian guys, so there had to be something gross and funny about
it. When they announced it for the first time in the ESO auditorium there
was an outburst of laughter and people roaring in their seats. This time
the name stuck. Not for lack of finding a better one, but because the
pattern had become a real tribute to <em>largeness</em>. Prepare more jokes.</p>
<p>I thought OWL was pretty cool and the project quickly got a mascot. Project
was never cancelled because it was never really started or achievable, it
would have bankrupted a few European countries for a too risky enterprise.
We kept the jokes though.</p>
<p>Fun to see it revived in an xkcd comic after all these years.</p>
One-time file-sharing2013-07-24T21:18:00+02:002013-07-24T21:18:00+02:00nicolas314tag:maz.one,2013-07-24:/one-time-file-sharing.html<img alt="One" src="/images/one.png" style="width: 300px; height: 300px;" />
<p>Say you rent a box somewhere on the Internet. You installed Debian stable on it because you want it to be nice and stable and run a few daemons that are useful to have online. Could be to hold your vast music collection, family pictures, or use it as remote …</p><img alt="One" src="/images/one.png" style="width: 300px; height: 300px;" />
<p>Say you rent a box somewhere on the Internet. You installed Debian stable on it because you want it to be nice and stable and run a few daemons that are useful to have online. Could be to hold your vast music collection, family pictures, or use it as remote storage for backup. Imagine you wanted to share some of the files hosted on this box with your relatives, who may or may not be computer-literate. Most of them would know how to use a webmail but asking them to install an ftp client is just beyond reach. Obviously, you do not want to give these guys too many rights over your box (like an ssh access for scp). What are the solutions?</p>
<p><strong>Setting up a dedicated HTTP server</strong></p>
<p>Simple enough: set up an HTTP server to distribute static files. lighttpd is simple enough to setup in a couple of minutes and is very efficient for static stuff. But you do not want to distribute your files to the whole Internet. Sooner or later a web spider will crawl in and index your family pictures and all sorts of things you never meant to be public. Next step: configure password-protection on the server</p>
<p>Fair enough. Now you have limited file downloads to people who know the password -- provided they know how to enter a password. Do you create multiple accounts, one for each of your peers? It would be preferrable, otherwise you will never know who downloaded what. But then you have to communicate their passwords to your peers and make sure they have a procedure in case they forget it. You know you are headed straight to massive butt pains.</p>
<p>Second issue: passwords can be shared. You shared that 2GB movie with a couple of friends and a couple of weeks later you find out that there are currently 1,549 active downloads for this file. Sharing is in human nature and that is completely Ok, but you probably did not sign up to become a content distributor over the whole Internet, only with a couple of friends and relatives.</p>
<p><strong>Next step: use one-time authentication</strong></p>
<p>There are better solutions out there: since you only mean to share one single file (or set of files) each time, you do not need to create accounts for your friends. You give them a one-time download token and forget about it.</p>
<p>A one-time download token is a URL. It looks like the kind of URLs you get from URL shorteners with the funny string at the end. Something like <em>http://shortener/12398741</em></p>
<p>One-time tokens can be shared but since they can only be used once, the person who shared it has lost it. The token is randomly generated and invalidated immediately after it is used to avoid having robots automatically scan all possible URLs in a row until they find a valid one.</p>
<p>There are many ways to achieve this on regular HTTP servers. Apache probably has a million configuration options for user authentication, including one-time passwords or something similar, but I have to admit I did not even try. I already wasted enough of my life in Apache config files. lighttpd can be configured to do that but the only solution I found required some Lua scripting and I did not feel up to the task.</p>
<p><strong>Next-step: Do It Yourself</strong></p>
<p>After reviewing countless pages of configuration options for various HTTP servers, I decided that it would be shorter for me to implement this in a tiny web app rather than try and understand complex configuration options. My first iteration made use of a Python FCGI script in web.py attached to a lighttpd process. Pointing out static files from a Python web app to the embedding lighttpd process is reasonably simple.</p>
<p>This implementation suffered from a number of pitfalls though. For one thing, performance was bad. For some reason, the Python process would eat insane amounts of CPU and RAM when sending big files, slowing down the server to a crawl. Second showstopper was the complexity involved for such a simple setup. I had to write a Python script to generate the lighttpd configuration file with a number of deployment options: where to put config files, log files, static files, port number, etc. And then came the inevitable issues with dependencies: Python version versus web.py version versus lighttpd version. Some combinations worked fine, some did not. Nothing specific to Python or lighttpd, but the more you have gears, the more you have places for grains of sand to fit in.</p>
<p>I still survived with this setup for a year or so, when Go came in. I have already reviewed the language in the past and will not come back to that, but suffice it to say that developing HTTP servers in Go is the most natural thing. Adding the one-time token ingredient to the soup was implemented in just one evening.</p>
<p>Once rewritten in Go, I found out that the end-result was about just as big as the Python implementation, excluding the script that created the lighttpd config. The main difference was of course that I do not have to maintain cross-references between package versions for Python, lighttpd, and web.py, since there is only one dependency to cover: Go itself.</p>
<p>It was straightforward to enhance the program to support more options, respond to favicon, and handle a JSON-readable database for active tokens. Performance is astounding. The serving Go process never takes more than a few megs of RAM (about the size of the executable itself) and only uses tiny amounts of CPU since the process is mostly I/O based anyway.</p>
<p>There is one thing I should have foreseen and had to re-implement. I am sending the one-time links by email and more and more people are reading their emails from their smartphone or tablet. Many just clicked the link without thinking twice, triggering a 2-4GB download and killing both their mobile and data plan at the same time. Wrong move.</p>
<p>The next version features a two-time download page: the first link sends users to a page summarizing the download, offering a second link to actually start the real thing with a big warning about the size of what will actually be sent.</p>
<p>There are many other features I would like to add to the current version, and I am hoping other people have better ideas for new features, which is the reason why I shared it on github. Find it here:</p>
<p><a class="reference external" href="https://github.com/nicolas314/onetime">https://github.com/nicolas314/onetime</a></p>
<p>Since we are talking about sharing private date between friends and relatives, protecting the download may be a good idea. A recently added feature was support for HTTPS. You only need to point your config to server certificate and key files and off you go. The HTTP/HTTPS thing is completely handled by Go.</p>
<p>The resulting program is far from top-quality but it fulfils the needs. Go give it a try if you want to. Careful though: it will only work on Linux boxes for now.</p>
Starbugs2013-06-26T00:39:00+02:002013-06-26T00:39:00+02:00nicolas314tag:maz.one,2013-06-26:/starbugs.html<p>The night was clear, we would not be playing Bomberman in the VLT control room that night. Clear skies and a sub-arcsecond <a class="reference external" href="https://en.wikipedia.org/wiki/Astronomical_seeing">seeing</a> meant we would have a full batch of data to process every hour or so until the next morning. Once the calibrations had finished, the telescope operator …</p><p>The night was clear, we would not be playing Bomberman in the VLT control room that night. Clear skies and a sub-arcsecond <a class="reference external" href="https://en.wikipedia.org/wiki/Astronomical_seeing">seeing</a> meant we would have a full batch of data to process every hour or so until the next morning. Once the calibrations had finished, the telescope operator launched the first observation. I re-compiled the whole processing software once more, just to be sure we had not forgotten anything, ran a series of unit tests for good measure, and waited in front of my screen for the first incoming set of frames to appear on the local disk.</p>
<p>First batch of sixty frames was completed after exactly sixty minutes. As the machine started doing its number-crunching, everybody in the room turned to me, waiting for the first processed image to come out. It took a good fifteen minutes for all algorithms to run through the set: calibrate all frames, remove the infrared sky, take into account bad and crazy pixels that have been hit with cosmic rays during the observation, register all frames to a common position, and finally stack them to a single image. The final result appeared on the screen above me and I could see smiles all around. It seemed the results were up to what my customers were expecting.</p>
<p>Now we had a clear image of a set of bright object against a dark background. In order to assess how much infrared light is emitted by each object, it needs to be calibrated. Somewhere on the image is a standard: a star with precisely known photometry in the wavelength we had been observing. Compute how many photons were received in this image from this star and you can deduce the magnitudes for all other objects present on the same frame.</p>
<p>I checked once more the final frame position on the sky and then launched the photometry calibration routine. The standard star was found and identified by name, its photometry computed by integrating all received light in a small surrounding radius, and then all objects in the frame were suddenly known by magnitude rather than number of photons. Perfect score! With a sigh of relief, I finally pushed myself away from the desk and reached for some water. The memory routines had done their job, we did not crash in flight by lack of RAM this time. Eleven more hours to go and then we could all go to sleep.</p>
<p>Next incoming data batch was processed just fine. Another image emerged. And then another one. It seemed everything was working perfectly fine.</p>
<p>Around midnight, something weird happened: the result image was correctly processed but photometry calibration failed because it found no standard star in the frame.</p>
<div class="line-block">
<div class="line">- What? Emilio, did you include a standard in the last observation?</div>
<div class="line">- Let me check... Yes I did. You should have it somewhere around the top-right corner.</div>
</div>
<div class="line-block">
<div class="line">The standard star was indeed there, so why did the photometry calibration routine fail to find it?</div>
<div class="line">I immediately opened the database we had for infrared standards and started searching frantically for the star, finding it immediately. I reached for the debugger and re-ran the whole routine once more with breakpoints. Confirmed: the search for standard stars in this region returned nothing, and yet the database was correctly loaded and completely in memory. The debugger showed what seemed like correct values for star positions, but the search function failed for some reason.</div>
</div>
<p>The star database we had was pretty simple: a simple text file containing named columns: first the star name, then its position on the sky as Right Ascension and Declination (a couple of angles), and then its magnitude at various wavelengths. Something like:</p>
<pre class="literal-block">
# Name | Ra | Dec | Sp | J | H | K
AS01-0 | 00 55 09.9 | 00 43 13 | -- | 10.716 | 10.507 | 10.470
AS03-0 | 01 04 21.6 | 04 13 39 | -- | 12.606 | 12.729 | 12.827
AS04-1 | 01 54 43.4 | 00 43 59 | -- | 12.371 | 12.033 | 11.962
AS05-0 | 02 30 16.4 | 05 15 52 | -- | 13.232 | 13.314 | 13.381
AS05-1 | 02 30 18.6 | 05 16 42 | -- | 14.350 | 13.663 | 13.507
AS07-0 | 02 57 21.2 | 00 18 39 | -- | 11.105 | 10.977 | 10.946
AS10-0 | 04 52 58.9 | -00 14 41 | -- | 11.349 | 11.281 | 11.259
AS13-1 | 05 57 10.4 | 00 01 38 | -- | 12.201 | 11.781 | 11.648
AS13-1 | 05 57 09.5 | 00 01 50 | -- | 12.521 | 12.101 | 11.970
AS13-3 | 05 57 08.0 | 00 00 07 | -- | 13.345 | 12.964 | 12.812
AS15-0 | 06 40 34.3 | 09 19 13 | -- | 10.874 | 10.669 | 12.628
AS15-1 | 06 40 36.2 | 09 18 60 | -- | 12.656 | 11.980 | 11.792
AS15-2 | 06 40 37.9 | 09 18 41 | -- | 13.711 | 12.927 | 12.719
AS15-3 | 06 40 37.9 | 09 18 19 | -- | 14.320 | 13.667 | 13.415
AS16-0 | 07 24 15.3 | -00 32 50 | -- | 14.159 | 14.111 | 13.305
AS16-1 | 07 24 14.3 | -00 33 05 | -- | 13.761 | 13.638 | 13.606
AS16-2 | 07 24 15.4 | -00 32 49 | -- | 11.411 | 11.428 | 11.445
AS16-3 | 07 24 17.2 | -00 32 27 | -- | 13.891 | 13.855 | 13.818
AS16-4 | 07 24 17.5 | -00 33 07 | -- | 11.402 | 11.106 | 11.043
</pre>
<p>J, H, K are infrared bands corresponding to a relatively narrow wavelength range.</p>
<p>Something went wrong in the star-loading routine, so I loaded the whole set into memory once more and dumped it back to a text file to plot it. The results were not particularly obvious:</p>
<img alt="Star catalog" src="/images/catalog.png" style="width: 300px; height: 225px;" />
<p>Somebody in the room came up to the screen and asked what we were looking at. I said: "these are the positions of all known infrared standards we have. For some reasons we cannot find tonight's star in here."</p>
<p>Looking at it again, I found our star. It was not in the right position. It should have been below the x axis but had shifted symmetrically above it. Looking at the data set again, the Declination was indeed negative: something like -00 14 41, but it was plotted on the wrong side of the x axis.</p>
<p>And then it dawned on me: the star was plotted at <strong>+00</strong> 14 41 instead of <strong>-00</strong> 14 41.</p>
<p>How do you read numeric data in C? Using scanf(). When you scanf() for "-00", what do you think ends up in memory? Zero. Positive zero, since it is technically the same as negative zero. Except the angle has now been flipped around the x axis.</p>
<p>Right: plotting a denser set of stars revealed a clear white patch for Declinations between zero and minus one. I had just forgotten to take into account the first character as a sign since scanf() does not make any difference between "00" and "-00". Once I corrected the database-loading line, everything fell into place and photometry computations could take place as expected.</p>
<p>Interestingly enough, it seems the same bug hit a large number of GPS devices over the past years. The German C't magazine told the story a few years back about somebody who planned a bike tour around Bordeaux and ended up with intermediate points in the middle of the ocean. Bordeaux is located around longitude zero (Greenwhich), so you do have data points located at an angle that starts with -00. In effect, you could see all points correctly plotted on the map except for the ones located between zero and minus one degree, which flipped over the other side of the meridian. As soon as I saw the map I knew exactly what had happened.</p>
<p>At least the guy was clever enough not to bike into high waters. It could have been worse: though probably related to time manipulation errors rather than angles, you may want to read how F22 Raptors spontaneously rebooted upon crossing the international date line:</p>
<p><a class="reference external" href="https://www.dailytech.com/Lockheeds+F22+Raptor+Gets+Zapped+by+International+Date+Line/article6225.htm">F22 Raptor gets zapped by international date line</a></p>
<p>There are some assumptions you should not make about handling time in software. Some of them are presented in this blog article:</p>
<p><a class="reference external" href="https://infiniteundo.com/post/25326999628/falsehoods-programmers-believe-about-time">Falsehoods programmers believe about time</a></p>
<p>Time and angles can be tricky scalars.</p>
Humans control Machines control Humans2012-11-30T11:16:00+01:002012-11-30T11:16:00+01:00nicolas314tag:maz.one,2012-11-30:/humans-control-machines-control-humans.html<img alt="Mind control" src="/images/mind-control.jpg" style="width: 247px; height: 204px;" />
<p>I used to work for a company where IT issues could only be reported by
email. No hotline: send an email, get a ticket number back, expect somebody
to call you about the issue and negotiate a solution with them. Of course,
it makes it a lot more fun to …</p><img alt="Mind control" src="/images/mind-control.jpg" style="width: 247px; height: 204px;" />
<p>I used to work for a company where IT issues could only be reported by
email. No hotline: send an email, get a ticket number back, expect somebody
to call you about the issue and negotiate a solution with them. Of course,
it makes it a lot more fun to report a broken email service or an
inaccessible network, but that's besides the point.</p>
<p>Email reporting could have worked if the Help Desk reaction times stayed
within reason but it sometimes took them days or even weeks before
answering your request. One of the IT guys carelessly gave me a tip one
day:</p>
<div class="line-block">
<div class="line">- Oh yeah, if your request is really urgent it should say so in the email you send.</div>
<div class="line">- You mean in the Subject or the Body?</div>
<div class="line">- Anywhere. The incoming filter puts it on top.</div>
</div>
<p>Gee... Let's try:</p>
<p>"Dear HelpDesk. I need more quota on my Unix account. Nothing urgent"</p>
<p>Sure enough, I got a call within the next minute and was immediately
granted more disk space.</p>
<p>I kept using this trick for a while and then somebody must have realized
they were being cheated. When <strong>urgent</strong> stopped working I switched to
<strong>this is not an emergency</strong>, which reached the expected result.</p>
<p>That is called gaming the system. Point is: once I knew my emails were
first read by a robot, I could influence their priority by choosing my
words accordingly. It got to a point where it would be pretty hard for a
human to determine what the reported problem was, but my email kept popping
up on top of the TODO list, which ensured a phone call from Help Desk
within the next minute.</p>
<p>So there you go: a human controls a piece of software that controls how
fast a human will respond to a human request.</p>
<p>We are doomed.</p>
That is an odd-smelling cheese2012-10-19T23:50:00+02:002012-10-19T23:50:00+02:00nicolas314tag:maz.one,2012-10-19:/that-is-an-odd-smelling-cheese.html<p><img alt="Cheese" src="/images/cheese-is-yellow.jpg" style="width: 300px; height: 305px;" />Just finished <a class="reference external" href="https://en.wikipedia.org/wiki/Who_Moved_My_Cheese%3F">Who moved my Cheese?</a> by Spencer Johnson. A quick and interesting read about how to be prepared for change and how embracing change in your life can make a real difference.</p>
<p>The followed metaphore is about four characters faced with a life-changing situation: they live in a maze …</p><p><img alt="Cheese" src="/images/cheese-is-yellow.jpg" style="width: 300px; height: 305px;" />Just finished <a class="reference external" href="https://en.wikipedia.org/wiki/Who_Moved_My_Cheese%3F">Who moved my Cheese?</a> by Spencer Johnson. A quick and interesting read about how to be prepared for change and how embracing change in your life can make a real difference.</p>
<p>The followed metaphore is about four characters faced with a life-changing situation: they live in a maze, they find a roomful of cheese and decide to live there. Time goes on, the cheese gets stale. One day the cheese is gone and they have to travel through the maze to find a new cheese room. Two characters move forward and find happiness further down the maze, one character refuses to move and ends up all alone and hungry. The last character evolves through the story: at first he is stuck in his comfort zone, then he learns to move forward and finally finds new cheese.</p>
<p>The story is short and like other self-help books, only filled with enthusiasm and positive thinking. The book fails in many ways by being so far off from reality. Change is good, but change can also be dangerous. The maze is not always just an endless stream of walls, it can also hide terrible dangers and taking the decision to leave your comfort zone could be fatal. The whole difficulty lies in knowing exactly when it is time to move on and when it is too early.</p>
<p>This reminded me of a <a class="reference external" href="https://en.wikipedia.org/wiki/Nasreddin">Nasreddin Hodja</a> story.</p>
<p>One day Nasreddin is awoken by his father at 5am. His father says: "Come on Nasreddin wake up! The early bird catches the worm! Come with me to the market, we will be the first ones there to sell our stuff".</p>
<p>But Nasreddin stays in bed and does not even acknowledge his father's presence. He insists: "Nasreddin, starting early is the first step on the path to wealth. Get up!"</p>
<p>Nasreddin does not move.</p>
<p>His father says: "Nasreddin, you remember when I left home at 5am the other day? On my way to the market, I found a purse filled with gold. If I had not been so early, somebody else would surely have found it before me."</p>
<p>Nasreddin opens an eye and says: "You know what? The guy who lost the purse woke up even earlier than you did." With that said, he goes back to sleep.</p>
<p>The story in <em>Who moved my cheese?</em> does not hold water but at least it forces you to think about change. How do you handle it? When do you realize you will soon need to change? When does the cheese start smelling odd enough that you want to move forward? To which you could add: is it better to keep gulping down stale cheese or get eaten by a cat?</p>
Die C++, Die2012-05-10T16:22:00+02:002012-05-10T16:22:00+02:00nicolas314tag:maz.one,2012-05-10:/die-c-die.html<div class="line-block">
<div class="line">Why should I have written ZeroMQ in C, not C++</div>
<div class="line"><a class="reference external" href="https://www.250bpm.com/blog:4">https://www.250bpm.com/blog:4</a></div>
</div>
<p>We now have official confirmation that the C++ standardization body is made of extra-terrestrial beings come to Earth to prevent humanity from reaching the singularity stage. By creating C++ and making it appear as …</p><div class="line-block">
<div class="line">Why should I have written ZeroMQ in C, not C++</div>
<div class="line"><a class="reference external" href="https://www.250bpm.com/blog:4">https://www.250bpm.com/blog:4</a></div>
</div>
<p>We now have official confirmation that the C++ standardization body is made of extra-terrestrial beings come to Earth to prevent humanity from reaching the singularity stage. By creating C++ and making it appear as "the language of tomorrow", they succeeded in stalling software engineering into a Stone Age it will never quite recover from.</p>
<p>The above post is just a late realization of that fact.</p>
PC revival2012-03-18T23:47:00+01:002012-03-18T23:47:00+01:00nicolas314tag:maz.one,2012-03-18:/pc-revival.html<p>One of my home PCs just got into trouble lately. Survival time after booting was a couple of minutes, after which the whole thing froze hard, forcing a manual power-off. Does not look like an OS failure, it would die with a bit more blue elegance. Opened the box, found …</p><p>One of my home PCs just got into trouble lately. Survival time after booting was a couple of minutes, after which the whole thing froze hard, forcing a manual power-off. Does not look like an OS failure, it would die with a bit more blue elegance. Opened the box, found a thick layer of dirt on the processor, vacuumed all of it, closed the box and... no boot at all, no beeps, nothing. Just power everywhere but no video signal, no booting beeps. No beeps during boot could mean either motherboard or processor are fried. I took off the RAM chips and booted again, this time triggering the <em>no-RAM-panic-on-boot</em>, which signaled that at least the motherboard was still running. I just inserted the RAM chips carefully and rebooted, to be greeted by the usual boot sequence. Seems I just displaced them while cleaning.</p>
<p>Processor running temperature usually ran between 55 and 90 degrees, it has now gone down to 35 degrees. Mental note: clean the computers at least once a year before they fry. And if you think the motherboard is gone, try booting it without processor or RAM to see if it can still complain.</p>
Daily Fallacies2011-10-26T23:12:00+02:002011-10-26T23:12:00+02:00nicolas314tag:maz.one,2011-10-26:/daily-fallacies.html<img alt="Monkey" src="/images/monkey-banana.png" style="width: 128px; height: 139px;" />
<p>To the question: "<em>Why do we do things this way?</em>", you often get three different kinds of answer:</p>
<dl class="docutils">
<dt><strong>Argument by Longevity</strong></dt>
<dd><em>"Because we have always done things this way"</em>, also known as <em>"It is known."</em></dd>
<dt><strong>Argument by Numbers</strong></dt>
<dd><em>"Because everybody does it this way, there must be a reason."</em></dd>
<dt><strong>Argument …</strong></dt></dl><img alt="Monkey" src="/images/monkey-banana.png" style="width: 128px; height: 139px;" />
<p>To the question: "<em>Why do we do things this way?</em>", you often get three different kinds of answer:</p>
<dl class="docutils">
<dt><strong>Argument by Longevity</strong></dt>
<dd><em>"Because we have always done things this way"</em>, also known as <em>"It is known."</em></dd>
<dt><strong>Argument by Numbers</strong></dt>
<dd><em>"Because everybody does it this way, there must be a reason."</em></dd>
<dt><strong>Argument by Authority</strong></dt>
<dd><em>"Because the experts say it must be done so."</em></dd>
</dl>
<p>These arguments are examples of <a class="reference external" href="https://en.wikipedia.org/wiki/Fallacy">fallacies</a>. A large number of examples and counter-examples are provided on the Wikipedia page, I will merely provide some here:</p>
<ul class="simple">
<li><em>Longevity</em>: Humanity has survived 100,000 years without need for <a class="reference external" href="https://en.wikipedia.org/wiki/Toothbrush">toothbrush</a>, therefore toothbrushes are useless for human existence.</li>
<li><em>Numbers</em>: Most people believe that if you toss a coin 10 times and heads come out 10 times, the next time you toss the coin it has less chances of coming up with heads. Therefore it must be true.</li>
<li><em>Authority</em>: My favourite singer votes for candidate X, therefore I will vote for candidate X.</li>
</ul>
<p>A fallacy derives from the fact that there is no link between the premises, leading to a wrongly acquired conclusion. A statistical fact is independant from belief, it is a mathematical truth that can be demonstrated. Adding more people or time to the fact does not influence the demonstration in any way. Somebody claiming to be a math expert declaring that the 11th toss has different than 50/50 chances could easily be proven wrong.</p>
<p>A science experiment on monkeys was apparently carried out in 1967 (<a class="reference external" href="https://wiki.answers.com/Q/Did_the_monkey_banana_and_water_spray_experiment_ever_take_place">Did the monkey banana and water spray experiment ever take place?</a>). The experiment could be summarized as:</p>
<p>Five monkeys were locked in a cage with a banana hanging from the ceiling. Whenever a monkey tried to get the banana they were all sprayed with ice-cold water. After a while they stopped trying. Next step: they replaced one of the monkeys. The newcomer tried to get to the banana but the others would beat him up before he had a chance, knowing perfectly well that touching the banana triggered a cold shower. The researchers kept replacing monkeys one by one until the ones left had never been sprayed with cold water but kept beating up any newcomer who would dare get close to the banana.</p>
<p>The story is often told to illustrate why large organizations tend to cristallize around age-old processes, even when they stopped making sense a while ago. In fact I have seen it happen in nearly every company I have ever visited, no matter how big or small, in a dozen countries in Europe, Africa, and Americas.</p>
<p>Who has never spent a half-hour filling up expense forms for sums largely inferior to what is actually spent in people's time filling up and processing these forms? Why should it be done otherwise? The rules are the same for one or a thousand euro expense, everybody has always done it this way and nobody ever complained. I know only one company who would pay systematic and fixed lumpsums for travel expenses. You only had to fill forms if you could justify spending more than the allowance.</p>
<p>Argumenting by authority is quite common in the workplace. An example would be external consultants who come up observing for a day or two, go back to their office and end up sending a large report describing how work processes should be changed without giving any other alternative or trying to argument their suggestions. I have seen that happen more times than I am willing to admit.</p>
<p>I do not believe an expert just because he declares himself such. Experts are knowledgeable people so what I expect from them are clear arguments, new data, and demonstrations. I need to follow the same path if I want to end up with the same conclusions. Of course, a talented and biased expert could only provide data pervasive to the point he is trying to make. This is another kind of fallacy and it is pretty hard to detect as soon as things get outside your own fields of expertise.</p>
<p>Fallacies are common, they are everywhere. We all do it because it is easier than a completely logical train of thought that requires mental exercise. They tend to convince people easily, especially when they end up with a correct conclusion. Example: "Most people these days brush their teeth, therefore it is a good thing. Experts will confirm this."</p>
<p>Yes, dentists will confirm it. And brushing your teeth is definitely good for your health. But you should do it because it has been proven that it helps you keep your real teeth longer and suffer less from cavities, not just because lots of people do it. Proven? Quite a bit: do a bit of search for yourself (hint: use the tubes, Luke), though this will be left as an exercise to the reader.</p>
<p>Coming back to my <em>field of expertise</em>, fallacies in software engineering are a dime a dozen.</p>
<ul class="simple">
<li>A large majority of developers can code in Java, therefore my project should be coded in Java.</li>
<li>Many programs I use daily are coded in C++, therefore my project should be coded in C++.</li>
<li>95% desktop users are running Windows, therefore I should use Windows</li>
<li>CORBA has been designed by a panel of experts, therefore CORBA is an expert-level technology, I should use it in my project.</li>
<li>Software is made of lines of code, therefore coding is all that matters in a software project. A programmer's productivity is measured in the number of lines s/he writes every day.</li>
</ul>
<p>A closely-related fallacious trait commonly found among young software engineers:</p>
<ol class="arabic simple">
<li>My program crashes</li>
<li>My code is bug-free</li>
<li>Therefore: the environment around my code is faulty</li>
</ol>
<p>I once coached an intern who told me: <em>"I double-checked my programs and there are no faults, but the binary crashes every time I launch it so the compiler must have introduced bugs in it."</em> Other young colleagues have found numerous bugs in interpreters, libraries, the operating system, and when all else fails: blame the incompetent user. Yep, all these things can have bugs, but maybe you should first look into the most recent element added to the system, i.e. your own code, before looking into other directions.</p>
<p>Not a day goes by without meeting a young engineer who lectures me about software engineering, usually answering my <em>"why do you do it this way?"</em> by <em>"this is the way we have always done software here"</em>. I am ready to accept any kind of debatable argument but not the longevity fallacy coming from people who have not defined the rules themselves. If things have always happened this way, what was the reason for it initially? Is it still valid today? Are we in the same context now? Why can't we touch the frigging banana?</p>
<p>The only lesson I got from this over time is: pick the banana, always. Best scenario: you now have a banana. Worst-case scenario: now you know exactly why nobody touched it, even though nobody could explain it to you before.</p>
Celebs on sex2011-10-21T22:56:00+02:002011-10-21T22:56:00+02:00nicolas314tag:maz.one,2011-10-21:/celebs-on-sex.html<p>Just saw this on <a class="reference external" href="http://imgur.com/a/UVlYC">Imgur</a>. For sake of completeness I will just quote them here:</p>
<div class="line-block">
<div class="line"><em>"Sex at age 90 is like trying to shoot pool with a rope."</em></div>
<div class="line"><em>-- George Burns</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"I believe that sex is one of the most beautiful, natural, wholesome things that money can buy."</em></div>
<div class="line"><em>-- Tom Clancy</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"You …</em></div></div><p>Just saw this on <a class="reference external" href="http://imgur.com/a/UVlYC">Imgur</a>. For sake of completeness I will just quote them here:</p>
<div class="line-block">
<div class="line"><em>"Sex at age 90 is like trying to shoot pool with a rope."</em></div>
<div class="line"><em>-- George Burns</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"I believe that sex is one of the most beautiful, natural, wholesome things that money can buy."</em></div>
<div class="line"><em>-- Tom Clancy</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"You know "that look" women get when they want sex? Me neither."</em></div>
<div class="line"><em>-- Steve Martin</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"Having sex is like playing bridge. If you don't have a good partner you'd better have a good hand."</em></div>
<div class="line"><em>-- Woody Allen</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"My girlfriend always laughs during sex -- no matter what she's reading."</em></div>
<div class="line"><em>-- Steve Jobs</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"Bisexuality immediately doubles your chances for a date on a Saturday night."</em></div>
<div class="line"><em>-- Rodney Dangerfield</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"My mother never saw the irony in calling me a son-of-a-bitch."</em></div>
<div class="line"><em>-- Jack Nicholson</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"Clinton lied. A man might forget where he parks or where he lives but he never forgets oral sex, no matter how bad it is."</em></div>
<div class="line"><em>-- Barbara Bush</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"Women need a reason to have sex. Men just need a place."</em></div>
<div class="line"><em>-- Billy Crystal</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"According to a new survey, women say they feel more comfortable undressing in front of men than they do undressing in front of other women. They say that women are too judgmental, where, of course, men are just grateful."</em></div>
<div class="line"><em>-- Robert de Niro</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"Instead of getting married again, I'm going to find a woman I don't like and just give her a house."</em></div>
<div class="line"><em>-- Rod Stewart</em></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><em>"See, the problem is that God gives men a brain and a penis, and only enough blood to run one at a time."</em></div>
<div class="line"><em>-- Robin Williams</em></div>
</div>
Config file hell2011-10-12T22:44:00+02:002011-10-12T22:44:00+02:00nicolas314tag:maz.one,2011-10-12:/config-file-hell.html<img alt="Complexity" src="/images/complexity.jpg" style="width: 300px; height: 112px;" />
<p><a class="reference external" href="https://xkcd.com/963/">xkcd 963</a> nailed it once again. How much fun is it to have to open one of the zillion Unix config files on your Debian box and start tweaking until it finally works? The graph could just as well show "Time since I last opened wpa_supplicant.conf", or "/etc/network …</p><img alt="Complexity" src="/images/complexity.jpg" style="width: 300px; height: 112px;" />
<p><a class="reference external" href="https://xkcd.com/963/">xkcd 963</a> nailed it once again. How much fun is it to have to open one of the zillion Unix config files on your Debian box and start tweaking until it finally works? The graph could just as well show "Time since I last opened wpa_supplicant.conf", or "/etc/network/interfaces", "fontconfig", "httpd.conf", or "crontab".</p>
<p>Unix is famous for its configurability. Unfortunately it has never offered a single convenient base library to support configuration file parsing. Every little piece of software had to design its own format, generate its own syntax rules (which complexity mostly depend on the programmer's talent for parser writing), and force you to learn yet another language that will take you hours to understand and minutes to forget. Examples:</p>
<div class="section" id="crontab">
<h2><a class="reference external" href="https://www.manpagez.com/man/5/crontab/">crontab</a></h2>
<p>Seems the syntax for crontab files is just a bit beyond what my brain can absorb. I have set millions of cron jobs in my life and still cannot write one without copy/paste from an existing file.</p>
</div>
<div class="section" id="sudoers">
<h2><a class="reference external" href="https://linux.die.net/man/5/sudoers">sudoers</a></h2>
<p>The associated man page does not just describe a set of options, it goes as far as defining a <em>full-fledged language</em> by providing a formal BNF grammar, as if end-users were yacc compilers. It even features significant whitespace. Yummy.</p>
</div>
<div class="section" id="procmailrc">
<h2><a class="reference external" href="https://userpages.umbc.edu/~ian/procmail.html">procmailrc</a></h2>
<p>Before we had spam filters, <tt class="docutils literal">procmail</tt> was the only wall between a sane inbox and a wave of unsollicited messages. But there was a heavy price to pay: learn how to write a bug-free <tt class="docutils literal">procmailrc</tt> with no way to test it except to send yourself half a million fake e-mails until you got it right. I still have a couple of procmailrc templates somewhere just in case I ever have to get into this again.</p>
</div>
<div class="section" id="sendmail-cf">
<h2><a class="reference external" href="https://www.sendmail.org/m4/readme.html">sendmail.cf</a></h2>
<p>Sendmail.cf will probably earn the gold medal of the most obscure, un-debuggable, impossible to write and just frankly insane configuration file there ever was. It is now mentioned in the Geneva convention about the non-proliferation of mental-illness-inducing configuration formats. But let us not be too harsh. Without it we would not have had <a class="reference external" href="http://www.ibiblio.org/harris/500milemail.html">the case of the 500-mile email</a> and a lot less horror stories to tell our children.</p>
<p>Hopefully things are getting better. Now that we have XML we can tear away the last shreds of hope of ever understanding how to configure a piece of software by editing a file. And if you are really vicious, you could go as far as <a class="reference external" href="http://stackoverflow.com/questions/4417458/how-come-there-is-no-apache-ant-xml-schema-definition-or-dtd-for-build-scripts">creating an XML-like config file format that cannot be validated</a>.</p>
<p>I once had a problem, then I discovered XML, and then I had two problems.</p>
</div>
Stormy Sunday2011-04-25T17:56:00+02:002011-04-25T17:56:00+02:00nicolas314tag:maz.one,2011-04-25:/stormy-sunday.html<img alt="Orcs vs Dwarves" src="/images/gimme-the-prize.jpg" style="width: 300px; height: 192px;" />
<p>My kids were bored on one of these unkind-weathery Sundays so we decided to invent a new war game. Creating the game from scratch took us most of the afternoon and we had to modify rules several times to make it balanced. This is what we ended up with.</p>
<div class="section" id="setup">
<h2>Setup …</h2></div><img alt="Orcs vs Dwarves" src="/images/gimme-the-prize.jpg" style="width: 300px; height: 192px;" />
<p>My kids were bored on one of these unkind-weathery Sundays so we decided to invent a new war game. Creating the game from scratch took us most of the afternoon and we had to modify rules several times to make it balanced. This is what we ended up with.</p>
<div class="section" id="setup">
<h2>Setup</h2>
<p>We picked a bunch of Lego characters, dwarves and orcs. I stumbled upon these in a toy store one day and got them for pretty much nothing, thinking they might be fun for the boys. Six dwarves will be fighting 12 orcs.</p>
<img alt="The proud dwarf team" src="/images/dwarf-team.jpg" style="width: 300px; height: 153px;" />
<img alt="Orc team" src="/images/orc-team.jpg" style="width: 300px; height: 205px;" />
<img alt="Game setup" src="/images/game-setup.jpg" style="width: 300px; height: 209px;" />
<img alt="Game setup (2)" src="/images/game-setup-2.jpg" style="width: 297px; height: 300px;" />
<p>For the board itself we used our carpet: a blue background with yellow dots organized along an orthogonal grid, perfect to align pieces. Walls are delimited with Kapla pieces because they happen to have the correct length to align with the carpet dots, but we could have used books, rulers, or larger pieces of Lego.</p>
<p>We created a maze with the wooden blocks, with long narrow corridors, wide rooms, choke points and two entrances. This was a lot more difficult than initially expected as paths need to be created to avoid making an obvious, shorter route to treasures.</p>
</div>
<div class="section" id="rules">
<h2>Rules</h2>
<p>Time to make up rules: the dwarves ought to be much stronger than the orcs because they are the good guys. So the orcs should be in infinite numbers to compensate, which pushes for a mission-based game where the six dwarves have a goal to achieve and countless orcs are trying to prevent them. We distributed three gold gems on the board: the dwarves have to pick them up and bring them all back to their entrance to win the game.</p>
</div>
<div class="section" id="moves">
<h2>Moves</h2>
<p>Each player plays in turn. We have an orthogonal grid so movements should be in all directions but diagonals. Dwarves can use up to 6 movement points each turn, orcs can use up to 8. Once a player has moved all his pieces he can attack.</p>
</div>
<div class="section" id="combat">
<h2>Combat</h2>
<p>Orcs have 1 attack, 1 defense, and 1 hit point. Once an orc has been hit it is dead and removed from the game.</p>
<div class="line-block">
<div class="line">Dwarves have each 2 attack, 2 defense and 2 hit points. The first time a dwarf is hit it looses its helmet or shield and reduces to 1 attack, 1</div>
<div class="line">defense and 1 hit point. Dwarves are killed on the second sucessful attack.</div>
</div>
<div class="line-block">
<div class="line">Attacks are solved using this table:</div>
</div>
<img alt="Resolution table" src="/images/lego-battles.png" style="width: 450px; height: 182px;" />
<div class="line-block">
<div class="line">Att-Def is the total attack points minus the defense points for the attacked piece, the leftmost column is the result of a tossed six-sided dice.</div>
</div>
<ul class="simple">
<li>A-1 means: attacker looses one hit point</li>
<li>D-1 means: defenser looses one hit point</li>
<li>AE means Attacker Eliminated, DE means Defender Eliminated</li>
</ul>
<p>Each piece may only attack once during its turn.</p>
<p>Several pieces may attack the same one, cumulating their attack points. The player who receives hit points decides how to distribute them to his pieces.</p>
<p>Building the game turned out to be much more fun than actually playing it. The first few games had to be re-balanced with everybody contibuting new rules and the final result is quite simple and satisfactory.</p>
</div>
<div class="section" id="beyond-orcs-and-dwarves">
<h2>Beyond Orcs and Dwarves</h2>
<p>Starting from there we created more Lego-based games around the same set of rules: Jedi knights against stormtroopers and clones, with special rules for heroes, long-range weapons, line of sight and buildings. The same rules were then extended for space battles: ships have several weapons, shields, and hit points corresponding to various parts of their hulls.</p>
<p>Opening the game to three players was a bit more difficult: if the three have equal power two of them will unite against the third, who will be immediately overwhelmed. The poor third player has then the privilege of choosing who of the remaining two will win, by concentrating attacks on one of the two players before he gets eliminated from the board. Our best bet was to spread power as 2 vs 1+1: one player has as much power as the other two combined, which brings the game back to a classic 1vs1.</p>
<p>And then Spring started showing up and it was much more fun to go rollerskating around the neighbourhood with friends. Can't say we miss stormy Sundays though.</p>
</div>
8-bit life2011-01-15T00:28:00+01:002011-01-15T00:28:00+01:00nicolas314tag:maz.one,2011-01-15:/8-bit-life.html<p>These two are a lot deeper than initially expected:</p>
<ul class="simple">
<li><a class="reference external" href="http://www.youtube.com/watch?v=SEVU-YLpM8A">Dan the Man (youtube)</a></li>
<li><a class="reference external" href="http://www.youtube.com/watch?v=_Mixd7ss83Y">Consoul - Lasse Gjertsen (youtube)</a></li>
</ul>
Printer Ink worth more than Human Blood2010-05-25T18:59:00+02:002010-05-25T18:59:00+02:00nicolas314tag:maz.one,2010-05-25:/printer-ink-worth-more-than-human-blood.html<p>HP has apparently started a new PR campaign trying to justify the insanely high prices for the ink they sell along their printers. I have not seen the PR yet but judging from the absolute tsunami of rage that has erupted in blogs today, it seems like a pretty bad …</p><p>HP has apparently started a new PR campaign trying to justify the insanely high prices for the ink they sell along their printers. I have not seen the PR yet but judging from the absolute tsunami of rage that has erupted in blogs today, it seems like a pretty bad move from HP.</p>
<p>Check out this discussion on slashdot:</p>
<p><a class="reference external" href="http://tech.slashdot.org/story/10/05/24/2258256/HP-Explains-Why-Printer-Ink-Is-So-Expensive">HP explains why printer ink is so expensive</a></p>
<p>And this enlightening blog post:</p>
<p><a class="reference external" href="http://thesharklady.com/tech/business/obviously-hp-thinks-we-are-stupid">Obviously HP thinks we are stupid</a></p>
<p>Everybody knows the story about printers and ink: give away the razor, sell the blades. As people are bound to buy new blades at regular intervals, the little money lost on the razor itself is re-couped a million times on expendables.</p>
<p>I have such a color printer at home: the HP kind, including a (bad) scanner and a fax function I will never use. I fell in for the cheap price -- the thing cost less than 50 euros in a sale -- and immediately regretted it. I could only print a mere 50 pages before I had to buy new ink for 25 euros, i.e. a printer is worth two ink cartridges. And when you put in a new cartridge, the printer automatically goes printing a <em>calibration page</em> that you are asked to scan to <em>calibrate the scanner</em>. When this happened I went crazy, started running around the house yelling after HP for getting me with such a cheap trick.</p>
<p>Ok, I have been screwed, I admit it.</p>
<p>But now HP going to their customers and trying to explain at length how they screw them is maybe a little too much.</p>
<p>Relevant:</p>
<p><a class="reference external" href="http://theoatmeal.com/comics/printers">http://theoatmeal.com/comics/printers</a></p>
<p>Other examples of crappy marketing taking over product quality?</p>
<div class="section" id="iphones-are-worthless">
<h2>iPhones are worthless</h2>
<p>Apple sells iPhones according to their mass storage capacity. A 32GB model costs twice as much as a 16GB model, but the only part that changes between both is a memory chip. Mathematically, if P is the price of the phone without any memory and M the price of a 16GB chip, we have:</p>
<pre class="literal-block">
2(P+M) = P+2M
</pre>
<p>If you work out the math you end up with</p>
<pre class="literal-block">
P=0
</pre>
<p>i.e. an iPhone is worthless.</p>
</div>
<div class="section" id="sms-are-a-rip-off">
<h2>SMS are a rip-off</h2>
<p>SMS prices: anything above zero is just greed from Mobile Operators. If you deploy a GSM network you have SMS capabilities built-in because they are part of the very network itself, they travel on the signalling network. Charging your users for SMS exchanges is like charging customers for the rental of all chairs and table in your restaurant.</p>
<p>BoingBoing published a post in 2008 explaining that <a class="reference external" href="http://boingboing.net/2008/05/12/sms-data-rate-is-4x.html">SMS data rate is 4 times more expensive than Hubble data</a>.</p>
</div>
<div class="section" id="diesel-powered-window-motors">
<h2>Diesel-powered window motors</h2>
<p>My car came with a number of options. One I did not take (200-300 euros worth) was an automatic window, i.e. you just press the button once and the window slides all the way to the bottom or the top (short-press again to stop the movement). The version I took only has this feature on the driver seat: for all other windows you need to maintain your finger on the button to move the window all the way up or down. Obviously this has nothing to do with hardware to install or configure, this is just a feature they disabled in the software handling window movements. But wait, there's more: they disabled a little too much and if you want to move any window but the driver's, the car engine has to be running. Yes: Renault invented the first diesel-powered window motors. This annoys me to no end.</p>
<p>I was told this kind of thing is called <em>negative marketing</em>. You degrade your product on purpose to attract customers to buy the "extra" features that have always been here but hidden on purpose. Now this is beyond me. The only message I get from that is: "we will screw you no matter what, stop fighting", nothing to do with product quality.</p>
<p>Curiously, this does not seem to jeopardize industrials who still think it is cool to milk their customers. The message I get from HP here is that they are only after the money and could not care less about their users. We are milk cows to these guys.</p>
</div>
Perl Syntax in one image2009-11-13T17:22:00+01:002009-11-13T17:22:00+01:00nicolas314tag:maz.one,2009-11-13:/perl-syntax-in-one-image.html<p>I recently had to switch back to Perl programming -- after 10+ years of carefully avoiding it. For some reason it brought back to my mind the kind of image found below. It took me 30 years to realize Franquin's characters are all Perl programmers.</p>
<img alt="Perl syntax" src="/images/jurons-gaston4.jpg" style="width: 450px; height: 293px;" />
Bugs? What bugs?2008-08-22T23:22:00+02:002008-08-22T23:22:00+02:00nicolas314tag:maz.one,2008-08-22:/bugs-what-bugs.html<p>The following story happened to me some years ago in a company I will not disclose to protect the innocent.</p>
<div class="line-block">
<div class="line">I was part of a group of users and we had been given a budget to spend on computer-related issues like getting new laptops, RAM or extra software. The group …</div></div><p>The following story happened to me some years ago in a company I will not disclose to protect the innocent.</p>
<div class="line-block">
<div class="line">I was part of a group of users and we had been given a budget to spend on computer-related issues like getting new laptops, RAM or extra software. The group would meet every 3 months to discuss how to spend the budget. During one of these meetings, the guy in charge asked around:</div>
<div class="line">"So all accounted for, we have about a thousand left. Any other ideas for potential purchases?"</div>
<div class="line">A girl raised her hand and said:</div>
</div>
<p>"Yes. I happen to be programming a lot in Fortran and I am having a hell of a time debugging because there we have no decent Fortran debugger available here. There is one available though, and it would cost about a thousand. Would that be something you would be willing to purchase?"</p>
<div class="line-block">
<div class="line">To which the guy promptly answered:</div>
<div class="line">"Debugger? To remove bugs?"</div>
<div class="line">"Well... yes."</div>
<div class="line">"No way. The easiest way is to do just like I do. Do not put bugs in your programs, that's all."</div>
</div>
<p>A long silence followed, glances were exchanged across the room and nobody knew if he was kidding us.</p>
<p>Apparently not, because he decided to close the session and waved off the Fortran debugger request with a sigh.</p>
<p>We later laughed our asses off about that one. Maybe we should have sent the guy around the world to teach how to make software that never has bugs, planes and cars that do not crash, food that is always tasty, you name it.</p>
<p>The story has more depth than this, though. One of the main principles in engineering is to steer away from all possible problems and defects when you design a system. So yes, putting less bugs in your software is definitely a trick of the trade. The only point he missed was that bug-free software simply does not exist, neither do crash-free planes or eternally tasty food. Go get a debugger.</p>