Regular expressions, or short "regex", are a pattern of characters and metacharacters that can be used for matching strings. For example, the pattern "gr[ae]y" matches both the strings "gray" and "grey".

While regular expressions are an integral part of other popular languages, they have been introduced to the Java world rather late with the release of Java 1.4 in 2002. Perl, certainly the mother language of modern regexes, already turned 15 that year.

Regexes are sometimes hard to understand, but once you got the hang of them, they will soon become your weapon of choice when you have to deal with texts.

In this article, I focus on Java code patterns for common scenarios. If you have never heard of regular expressions before, the Wikipedia article and the Pattern JavaDoc are good starting points. The Regex Crossword site is a great place for working out your regex muscles.

If you own a Synology NAS, you probably know the trouble that it frequently wakes up from hibernation without any obvious reason. Synology has added a more or less undocumented debugger for that issue. If enabled, it will log the reason for waking up from its beauty sleep, so one can take measures against it.

WARNING: This article is targeted at experienced Linux users. Even if you manage to enable the debug mode, you still have to analyze the log files, find a solution to your individual problem, and assess if it is safe to apply. If you make a mistake, you can cause a lot of damage to your system and your data. Please take this advice seriously.

In any case, make sure your data is properly backed-up. Remember that a RAID is not a backup!

Also note that the debug mechanism has changed from time to time, so it may be completely different on your NAS depending on the DSM version that is used.

I recently found an article about the AS3935 Franklin Lightning Sensor. As I am already recording some weather data, it immediately raised my interest.

The sensor module can be found at many online shops selling products from China. It is not really cheap, but still affordable. I decided to use an ESP8266 as microcontroller, so I can read the sensor data by WLAN. The sensor is connected to the ESP via SPI. There was also some space left for a SK6812 RGBW LED indicating the sensor status.

The result of this project can be found at GitHub. It's called Kaminari (which is Japanese for lightning), and also comes with OpenSCAD files for a 3D printed, pyramid shaped case with illuminated tip. In this article I will explain a bit about how I developed the Kaminari firmware.

The first problem was the calibration. The sensor is roughly pre-calibrated, but must be fine-tuned to 500 kHz ±3.5% via the TUN_CAP register. For this purpose, the antenna frequency can be routed to the IRQ pin and then be measured by the ESP. I chose to prescale the frequency by a ratio of 128, giving an IRQ frequency of 3,906.25 Hz. For measurement, I've set an IRQ handler that is just counting up a variable on each interrupt. I then reset the counter, wait for 1000 ms, then read the counter, and get the IRQ frequency in Hz units. It's not 100% accurate, but good enough for this purpose.

The TUN_CAP register offers 16 calibration steps. Just incrementing it until the frequency matches, would take up to 16 seconds. Instead I used an algorithm called successive approximation to find the correct calibration value in only 4 iterations, taking a quarter of the time.

To my disappointment, it turned out that the manufacturer of my module (CJMCU) has used nonstandard components, so my module could only reach a maximum frequency of about 491 kHz. I first suspected that the ESP might be too slow for this kind of measurement, but a scope connected to the IRQ pin confirmed the frequency. Well, it is still within the required tolerance, but it gives a suboptimal tuning result and renders the TUN_CAP register useless.

The next problem is finding a good noise floor level. This is some kind of background radio noise filter. If the level is too low, the sensor cannot operate properly because of interfering background noise. If it is set too high, the lightning detection quality declines.

The noise floor level cannot be calibrated just once at the start. Radio noise sources come and go, may it by turning on an electronic device or just by a change in the weather. I did some experiments, and the most promising solution is a kind of tug-of-war. When the AS3935 detects too much noise, it triggers an interrupt, and the noise floor level is raised to the next higher step. If the last level change was 10 minutes ago, the ESP attempts to reduce the level by one step.

In order to reduce the number of level changes, I have added a counter. Each noise interrupt increments the counter, and every 10 minutes the counter is decremented. The level is raised when the counter reaches 2, and lowered when the counter reaches -2.

Sometimes I noticed a "noise level runaway", where the AS3935 triggers a lot of noise interrupts in a very short time, raising the noise floor level to its maximum value immediately. To stop that behavior, further noise interrupts are being ignored for one minute after a noise interrupt has been processed.

Now the noise floor level has settled to an average of 95 µVrms here. In the graph, one can see that the level is raised at some time, but then reduced again after a while. One can also see the frequent attempts to lower the level a bit further, immediately followed by a raise back to the average level. It seems that the AS3935 and the ESP have negotiated a good compromise. 😉

The AS3935 seems to be set up in an optimal way now, but I still get some false lightning events from time to time. There are a few registers left to experiment with, namely WDTH (watchdog threshold), SREJ (spike rejection) and MIN_NUM_LIGH (minimum number of lightning). I have raised the watchdog threshold to 2, and did not have a false lightning event since then.

Now I have to wait for some real lightnings… 😄

The shredzone is celebrating its 20th birthday today! 🎂

Actually, my personal homepage is much older. It had started some day around 1995, when I was learning HTML. My first website was just a bunch of hand-made static pages. They were published on the free webspace of my ISP.

In 1998 I recreated my homepage, and called it "shredzone" for the first time. It was still using static pages, but now they were generated by running some scripts on my Amiga (there is a separate article about that if you're interested). This new site got bigger and bigger, and at some point ran out of quota on my ISP's webspace.

On April 15th 2000, I moved my site to a professional web hosting provider. I also bought my very first domain shredzone.de on that day. Now, having content and also a dedicated domain, the shredzone was finally complete!

In the coming years, I was switching from Amiga to Linux, and I was learning a lot of JavaScript and PHP. In 2003, I redesigned the website from scratch again. It was now using a self-made Content Management System called Akiko. A cool feature of Akiko was that I could use different seasonal page templates. On Christmas there was snow and a snowman, while on Halloween the pages were decorated with pumpkins and a manga witch that I had drawn myself.

Some years later, blogs were getting popular. I quickly added a blog feature to Akiko, and was learning about writing my own weblog since then. I found out that this format was much more useful for me, much better than the tree-structured contents I had before.

Soon I had reached the limits of Akiko. Writing a new blog article was tedious, especially when pictures were involved, so I badly neglected my blog for a couple of months. It couldn't go on like that, and I decided to write my own open source blog system from scratch. It is called Cilla and is written in Java now. Just in time for the 10th birthday, it was finally ready for prime time. My new weblog had a sleek and modern design, with a random photo on the top of each page. And it was much easier to use. Some of the old contents were migrated to my new blog, while many other outdated (or embarassing 😅) stuff was just dumped. But still my very first blog record goes back to the beginning of Akiko in 2003.

I liked the new design. However, it was cluttered with all kind of extras that have been modern when the blog scene started (like a calender and a tag cloud). Also, mobile devices have gotten ubiquitous over the years, but the old design was not optimized for them and just looked ugly on a small display.

So, in 2018 there was another major redesign, the one you are seeing now. It still runs on Cilla, but everything unnecessary has been removed, and some other necessary stuff like the comments fell victim to GDPR. On the other hand, it has a responsive design now.

Looking back these 20 years, it was a very interesting time. I have learned a lot about the internet, about websites, programming, and blogging. Now I'm curious what the Shredzone is going to look like in 20 years.

A minor downside of the Raspberry Pi is that it is not equipped with a battery backed-up real-time clock. After every reboot, the system time is messed up and needs to be corrected by NTP, which in turn requires a network connection.

Luckily, there are readily assembled RTC modules available. They base on the DS3231 real time clock chip. A tiny battery is keeping the time when the Raspberry is disconnected from power. You can find those modules for less than two Euros a piece at marketplaces like Amazon, eBay, or Alibaba. The module is just plugged onto the pin header of the RasPi.

It is quite easy to use the RTC on Raspbian. On Fedora for Raspberry Pi, the installation was a little more tricky though.

In a first step, the RTC must be added as a new I²C device:

/usr/bin/echo ds3231 0x68 > /sys/class/i2c-adapter/i2c-1/new_device

(If you have a Raspberry Rev 1, you'd use /sys/class/i2c-adapter/i2c-0/new_device instead.)

The RTC does not store the time zone, so we need to tell the system that we'd like to use the system's time zone:

timedatectl set-local-rtc 0

And finally we copy the current system's time to the RTC chip:

hwclock -w

The RTC is now set up and ready for operation. But we're not done yet. When the system boots up, the DS3231 is unknown to the system again. We have to add a systemd service for adding it and reading the time, by creating a file called /etc/systemd/system/my-rtc.service with the following content:

[Unit]
Description=Enable battery backed-up RTC
Before=basic.target
After=sysinit.target
DefaultDependencies=no

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStartPre=-/usr/bin/bash -c '/usr/bin/echo ds3231 0x68 > /sys/class/i2c-adapter/i2c-1/new_device'
ExecStartPre=-/usr/bin/sleep 0.2
ExecStart=/usr/sbin/hwclock -s

[Install]
WantedBy=basic.target

The service is enabled via:

systemctl enable my-rtc

And now every time the system boots up, the DS3231 is added as I²C device and the system clock is set to the time found in the RTC. If a network is available, NTP will later take over and set the network time.

It's not the most elegant solution, I guess. I had to add a sleep command because it turned out that the hardware is not immediately available after adding the device. I'd like to hear from you if you found a better way.

Remember to manually use hwclock -w from time to time, to reset the RTC to the correct time. If you shut down your RasPi frequently, you could add another systemd service that automatically writes the current time on system shutdown.