I recently found an article about the AS3935 Franklin Lightning Sensor. As I am already recording some weather data, it immediately raised my interest.

The sensor module can be found at many online shops selling products from China. It is not really cheap, but still affordable. I decided to use an ESP8266 as microcontroller, so I can read the sensor data by WLAN. The sensor is connected to the ESP via SPI. There was also some space left for a SK6812 RGBW LED indicating the sensor status.

The result of this project can be found at GitHub. It's called Kaminari (which is Japanese for lightning), and also comes with OpenSCAD files for a 3D printed, pyramid shaped case with illuminated tip. In this article I will explain a bit about how I developed the Kaminari firmware.

The first problem was the calibration. The sensor is roughly pre-calibrated, but must be fine-tuned to 500 kHz ±3.5% via the TUN_CAP register. For this purpose, the antenna frequency can be routed to the IRQ pin and then be measured by the ESP. I chose to prescale the frequency by a ratio of 128, giving an IRQ frequency of 3,906.25 Hz. For measurement, I've set an IRQ handler that is just counting up a variable on each interrupt. I then reset the counter, wait for 1000 ms, then read the counter, and get the IRQ frequency in Hz units. It's not 100% accurate, but good enough for this purpose.

The TUN_CAP register offers 16 calibration steps. Just incrementing it until the frequency matches, would take up to 16 seconds. Instead I used an algorithm called successive approximation to find the correct calibration value in only 4 iterations, taking a quarter of the time.

To my disappointment, it turned out that the manufacturer of my module (CJMCU) has used nonstandard components, so my module could only reach a maximum frequency of about 491 kHz. I first suspected that the ESP might be too slow for this kind of measurement, but a scope connected to the IRQ pin confirmed the frequency. Well, it is still within the required tolerance, but it gives a suboptimal tuning result and renders the TUN_CAP register useless.

The next problem is finding a good noise floor level. This is some kind of background radio noise filter. If the level is too low, the sensor cannot operate properly because of interfering background noise. If it is set too high, the lightning detection quality declines.

The noise floor level cannot be calibrated just once at the start. Radio noise sources come and go, may it by turning on an electronic device or just by a change in the weather. I did some experiments, and the most promising solution is a kind of tug-of-war. When the AS3935 detects too much noise, it triggers an interrupt, and the noise floor level is raised to the next higher step. If the last level change was 10 minutes ago, the ESP attempts to reduce the level by one step.

In order to reduce the number of level changes, I have added a counter. Each noise interrupt increments the counter, and every 10 minutes the counter is decremented. The level is raised when the counter reaches 2, and lowered when the counter reaches -2.

Sometimes I noticed a "noise level runaway", where the AS3935 triggers a lot of noise interrupts in a very short time, raising the noise floor level to its maximum value immediately. To stop that behavior, further noise interrupts are being ignored for one minute after a noise interrupt has been processed.

Now the noise floor level has settled to an average of 95 µVrms here. In the graph, one can see that the level is raised at some time, but then reduced again after a while. One can also see the frequent attempts to lower the level a bit further, immediately followed by a raise back to the average level. It seems that the AS3935 and the ESP have negotiated a good compromise. 😉

The AS3935 seems to be set up in an optimal way now, but I still get some false lightning events from time to time. There are a few registers left to experiment with, namely WDTH (watchdog threshold), SREJ (spike rejection) and MIN_NUM_LIGH (minimum number of lightning). I have raised the watchdog threshold to 2, and did not have a false lightning event since then.

Now I have to wait for some real lightnings… 😄

The shredzone is celebrating its 20th birthday today! 🎂

Actually, my personal homepage is much older. It had started some day around 1995, when I was learning HTML. My first website was just a bunch of hand-made static pages. They were published on the free webspace of my ISP.

In 1998 I recreated my homepage, and called it "shredzone" for the first time. It was still using static pages, but now they were generated by running some scripts on my Amiga (there is a separate article about that if you're interested). This new site got bigger and bigger, and at some point ran out of quota on my ISP's webspace.

On April 15th 2000, I moved my site to a professional web hosting provider. I also bought my very first domain shredzone.de on that day. Now, having content and also a dedicated domain, the shredzone was finally complete!

In the coming years, I was switching from Amiga to Linux, and I was learning a lot of JavaScript and PHP. In 2003, I redesigned the website from scratch again. It was now using a self-made Content Management System called Akiko. A cool feature of Akiko was that I could use different seasonal page templates. On Christmas there was snow and a snowman, while on Halloween the pages were decorated with pumpkins and a manga witch that I had drawn myself.

Some years later, blogs were getting popular. I quickly added a blog feature to Akiko, and was learning about writing my own weblog since then. I found out that this format was much more useful for me, much better than the tree-structured contents I had before.

Soon I had reached the limits of Akiko. Writing a new blog article was tedious, especially when pictures were involved, so I badly neglected my blog for a couple of months. It couldn't go on like that, and I decided to write my own open source blog system from scratch. It is called Cilla and is written in Java now. Just in time for the 10th birthday, it was finally ready for prime time. My new weblog had a sleek and modern design, with a random photo on the top of each page. And it was much easier to use. Some of the old contents were migrated to my new blog, while many other outdated (or embarassing 😅) stuff was just dumped. But still my very first blog record goes back to the beginning of Akiko in 2003.

I liked the new design. However, it was cluttered with all kind of extras that have been modern when the blog scene started (like a calender and a tag cloud). Also, mobile devices have gotten ubiquitous over the years, but the old design was not optimized for them and just looked ugly on a small display.

So, in 2018 there was another major redesign, the one you are seeing now. It still runs on Cilla, but everything unnecessary has been removed, and some other necessary stuff like the comments fell victim to GDPR. On the other hand, it has a responsive design now.

Looking back these 20 years, it was a very interesting time. I have learned a lot about the internet, about websites, programming, and blogging. Now I'm curious what the Shredzone is going to look like in 20 years.

A minor downside of the Raspberry Pi is that it is not equipped with a battery backed-up real-time clock. After every reboot, the system time is messed up and needs to be corrected by NTP, which in turn requires a network connection.

Luckily, there are readily assembled RTC modules available. They base on the DS3231 real time clock chip. A tiny battery is keeping the time when the Raspberry is disconnected from power. You can find those modules for less than two Euros a piece at marketplaces like Amazon, eBay, or Alibaba. The module is just plugged onto the pin header of the RasPi.

It is quite easy to use the RTC on Raspbian. On Fedora for Raspberry Pi, the installation was a little more tricky though.

In a first step, the RTC must be added as a new I²C device:

/usr/bin/echo ds3231 0x68 > /sys/class/i2c-adapter/i2c-1/new_device

(If you have a Raspberry Rev 1, you'd use /sys/class/i2c-adapter/i2c-0/new_device instead.)

The RTC does not store the time zone, so we need to tell the system that we'd like to use the system's time zone:

timedatectl set-local-rtc 0

And finally we copy the current system's time to the RTC chip:

hwclock -w

The RTC is now set up and ready for operation. But we're not done yet. When the system boots up, the DS3231 is unknown to the system again. We have to add a systemd service for adding it and reading the time, by creating a file called /etc/systemd/system/my-rtc.service with the following content:

[Unit]
Description=Enable battery backed-up RTC
Before=basic.target
After=sysinit.target
DefaultDependencies=no

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStartPre=-/usr/bin/bash -c '/usr/bin/echo ds3231 0x68 > /sys/class/i2c-adapter/i2c-1/new_device'
ExecStartPre=-/usr/bin/sleep 0.2
ExecStart=/usr/sbin/hwclock -s

[Install]
WantedBy=basic.target

The service is enabled via:

systemctl enable my-rtc

And now every time the system boots up, the DS3231 is added as I²C device and the system clock is set to the time found in the RTC. If a network is available, NTP will later take over and set the network time.

It's not the most elegant solution, I guess. I had to add a sleep command because it turned out that the hardware is not immediately available after adding the device. I'd like to hear from you if you found a better way.

Remember to manually use hwclock -w from time to time, to reset the RTC to the correct time. If you shut down your RasPi frequently, you could add another systemd service that automatically writes the current time on system shutdown.

In the first part I showed how the Sinclair ZX Spectrum stored data on tape. This second part explains what is stored, and what causes a tape loading error.

The ZX Spectrum BASIC offers a SAVE command for saving all kind of data. It can be used to save a BASIC program, variable arrays, but also arbitrary parts of memory. These files are always saved in two separate blocks. The first block is called header. It contains the file name, data type, and other meta information. The second block follows about a second later and contains the data itself.

The internal structure of each block is identical. The first byte distinguishes between header ($00) and data blocks ($FF). The final byte is a parity checksum. Everything between these two bytes is the payload.

A header block always contains a payload of 17 bytes. The first byte identifies the file type, followed by the file name (10 characters), followed by the length of the data block, and closed by two optional parameters that have different meanings depending on the file type. The length and the two parameters consume two bytes each, with the lower byte coming first because the Z80 CPU is little endian.

This is an example header block of a screenshot:

A screenshot is actually just a memory dump that starts at address $4000 (which is the starting address of the screen buffer) and is exactly 6912 bytes long (the ZX Spectrum has a resolution of 256×192 monochrome pixels plus 32×24 bytes color attributes, giving a screen buffer size of 6912 bytes).

For other file types, the two optional parameters have different meanings. For example, a BASIC program file stores the line number to start at after loading.

The final byte is the parity. It is used for error detection, and computed just by XOR-ing all the bytes that have been read. The result must be $00, otherwise a "R Tape loading error" is reported.

This kind of error detection is rather weak. Due to the nature of the XOR operation, two wrongs give a right. This means that when the block contains an even number of bad bits at the same position, they will be undetected. It is also not possible to correct reading errors, as the XOR operation only allows to identify the position of the bad bit, but not the actual byte that contained the error. More sophisticated error correction algorithms would have slowed down the loading process, though.

The parity is computed as a final step, after all the bytes have been read from the block on tape. For that reason, the loader can only decide at the end of the recording whether the loading was successful or not.

But then, why does the tape loading error sometimes appear while the block is still loading? Well, in the first part I have explained that the loading routine just reads an unknown number of bytes. It ends when waiting for a pulse change took to long. Now, if there is an audio gap on tape, the signal seems to end just in the middle of the block. It is then very likely that the parity checksum is wrong because there are still bytes missing.

Some simple copy protections made use of the way the Spectrum loads data from tape. A very common way were “headerless” files, where the header block was left out and only the data block was recorded on tape. The BASIC LOAD command was unable to read those files because of the missing header.

In the early time of home computers, at the beginning of the 1980's, hard disks and even floppy disks were too expensive for home use. The cheapest way for storing large amounts of data was the cassette tape. Cassettes and tape recorders were affordable and available in almost any household.

In this blog article, I'm going to explain how the Sinclair ZX Spectrum stored programs on cassette tapes. Other home computers of that time, like the Commodore 64 or Amstrad CPC, worked in a similar fashion.

Cassette tapes were designed to store audio signals like voice or music, so the inventors of the home computers had to find a way to convert data to audio signals. The easiest way is to serialize the data to a bit stream of 1's and 0's, and generate a long rectangular wave cycle for "1" and a short rectangular wave cycle for "0". This is what the ZX Spectrum actually does!

A short wave cycle is generated by giving power to the audio output for 855 so called T-states, and then turning off the power for another 855 T-states. A "T-state" is the time of a single clock pulse of the Z80-A CPU. As the CPU of a classic ZX Spectrum is clocked with 3.5 MHz, a T-state has a duration of 286 ns. The duration of a short wave cycle is thus 489 µs, giving an audio frequency of about 2,045 Hz. The long wave cycle is just twice as long.

Due to all kind of filters in the analog audio path, the rectangular signal is smoothed to a sinusoidal signal when played back. A Schmitt trigger inside the ZX Spectrum's hardware converts the audio signal back to a rectangular shape. Since the audio signal can have different amplitudes or could even be inverted, the hardware only cares for signal edges, not for levels. All that the loader routine now has to do is to measure the duration of the pulses, regenerate the bit stream, and put the bytes back together.

If you think that things cannot be that easy, you are right. 😄 The most difficult part for the loader is to find the start of the bit stream. If it is off by only one cycle (or even just a pulse), all the bytes are shifted by one bit, and the result is useless. All kind of noise on the tape makes it impossible to just wait for the signal to start, though.

For this reason, the recording starts with a leader signal, followed by a sync wave cycle, followed by the bit stream itself. The leader signal is just a continuous wave with a pulse length of 2,168 T-states, giving an 806 Hz tone that is displayed by red and cyan border colors on the TV. The sync wave cycle is a pulse of 667 T-States "on", followed by 735 T-states "off". After that, the actual data stream begins, which is displayed in blue and yellow border colors. When the last bit was transmitted, the data stream just ends.

So when the ZX Spectrum loads a file from tape, it first waits for the 806 Hz leader signal. If it was detected for at least 317 ms, it waits for the sync pulses, then it starts reading the bit sequence until there is a timeout while waiting for the next pulse.

It is a very simple way to store data on tape. And still, it is surprisingly reliable. After 30 years, I could recover almost all files from my old cassette tapes. Some of them were of the cheapest brands I could get my hands on back in 1987.

The only disadvantage is that this method is very slow. With 489 µs for a "0" and 978 µs for a "1", saving just 48 KBytes of data can take up to 6 minutes, giving an average bit rate of 1,363 bps (yes, bits per second). If we were to save a single 3 MBytes mp3 file that way, it would take almost 5 hours (and 5 cassettes with 60 minutes recording time each).

Some commercial games used speed loaders and copy protections. Speed loaders just reduced the number of T-states for the pulses, which increased the bit rate. Some copy protections used a "clicking" leader tone, where the leader signal was interrupted before the minimal detection time of 317 ms was reached. The original loader routine could not synchronize to these kind of signals, so it was impossible to read those files into copy programs. Those protection measures could still be circumvented by copying directly from tape to tape, but this only worked a few times due to increasing audio noise.

In the next article, I will take a deeper look at the bit stream contents, and I will also explain where the dreaded "R Tape loading error" comes from.