Nextcloud | David's Blog

Home alarm clock update

I am delighted with my current home alarm clock setup.

I have the Morefine M8S N100 Alder Lake PC running Fedora Workstation KDE Spin. It is cabled into an OLED television set. I use the remote on the TV: settings > energy saving > panel > turn the panel completely off.

The Morefine M8S stands out in that crowd of machines for having active cooling, but running quiet. At the moment, they are on sale with 12 GB RAM and a 128 GB NVME drive for $170.

A full kit Raspberry Pi isn’t that much less, doesn’t have full size HDMI, isn’t running Intel hardware (super Linux compatible) and they do have that nasty problem of writing /var/log to the SD card, burning it out. The newer Pi can do an NVME drive, but that’s going to drive the cost up close to the M8S. But I digress.

The tuner part of the TV stays on all the time. The M8S is cabled to the TV via an HDMI cable. Although the OLED panel is off, the tuner keeps the HDMI port active, so the M8S doesn’t disconnect or try to rejigger the display when the panel is turned off for sleep mode.

Previously, to connect to the sound bar, I used Bluetooth, bypassing the TV. This was not great. Specifically, Bluetooth wants to disconnect if not streaming, and Linux doesn’t want to play with an audio source that isn’t there. Sure, there are kludgy ways to get it to light up for a sound, but this is clearly paddling upstream for drudgery and debt.

But I saw the Toslink optical S/PDIF connector on the sound bar, and the lightbulb went on. Feed everything in to the TV over HDMI, and feed the audio from the TV to the sound bar over Toslink. We’re back to fun and profit. Well, fun, at least.

This is working out great.

The M8S is a Nextcloud client, so it has access to my audio files directory, and via the web, the Music app player.

KAlarm on the M8S runs VideoLAN Client (VLC) headless, pointing at the audio file (or for waking up, a .pls playlist file). I have seven .pls files for seven days of the week.

I also happen to have my source files for iPhone ringtones in Nextcloud, so when I want a short alarm sound, I’ve got a passel of those available.

KAlarm lets me set schedules for every Sunday of the month. I happen to need a different schedule for the second Sunday, where the first, third, fourth, and fifth are a normal schedule.

What is super nice is that I can have Nextcloud playing music at a low volume, but when VLC kicks in, it plays at the normal volume. They don’t interrupt each other, they simply multiplex.

Lastly, I’ve been playing with Coqui TTS (text-to-speech), so sometimes the sound being played by KAlarm is a feminine Scottish voice saying “Do daily inventory”. 🙂

Next, I want to generate Westminster Chimes sound files, and have it play those on the hour. It would be fun to generate the sound file on-the-fly, starting with the announcement chime and then Coqui TTS saying “Seven” or “Noon” or “Thirteen hundred”.

A future project I have in mind is to write a Perl script to pull down my calendar out of Nextcloud, and then write entries into the KAlarm file for special events.

Also, I wouldn’t mind turning the M8S (and my other machines) into Snapcast clients.

Fun indeed.

AI is getting good

Perplexity AI is proving to be a much better search engine than Google. It is astonishing.

For several years now, Google has been shooting themselves in the foot by trying to reform society through tainting search results. Accordingly, their search results have gone to shit. There are numerous examples of A/B tests against Google search: getting uplifting / supportive results when the female gender is the search, but getting condemning / demoralizing results when the male gender is the search. Ditto A/B test searches for Democrats versus Republicans, and Hillary Clinton versus Donald Trump.

Okay, I’m done: I have replaced my search engine in all my browsers with Perplexity AI.

So while Google was going to shit, Large Language Models became capable. Jeff Bezos of Amazon spun up his own, trained it, and is now putting it out there as Perplexity AI. They have a commercial license for $20 per month, which is too rich for my blood. If they had a $5 per month plan, I’d pay for it now.

But currently, I’m freeloading. I do pay for Twitter, so I might start using Grok (Grok3 just came out) instead. I prefer to spread my activity over different services. Yet, I feel that freeloading is something I don’t like in other people; therefore, if I want to live a high integrity life, I shouldn’t be a freeloader. Anyway….

Today I get to do the minutes for a monthly meeting that I am Recording Secretary for.

On my Windows machine that plays nice with the Sony ICD-UX570 Digital Voice Recorder, I uploaded the recorded MP3 file to Nextcloud. It syncs to the server, and the file ends up in my folder where all these recorded MP3s go. That folder syncs to my main machine I’m working on now.

I had that moment of inspiration that finding these files / messing with the folders is more trouble than it should be. I’m in these folders a lot. Windows has a “Favorites” feature, surely KDE has such too?

Asking of Perplexity AI “does KDE Dolphin have a Favorites feature” instantly took me to a page which presented a ZD Net article “How to use KDE Plasma Places for a much more efficient desktop“. This is precisely what I was looking for, yet I had no idea that it was called Places.

This is great. Google search might have gotten me to this page eventually, but I doubt that I’d have gotten to it without going through many different page views (thereby increasing Google’s ad revenue).

And yes, I’ve asked Perplexity AI to generate a Perl program for reading a Nextcloud calendar, and it took no time at all to do it. Now, the program did not work….

But troubleshooting the problem with Perplexity AI was pretty easy. It suggested I try curl and that worked perfectly.

As it turns out, Nextcloud doesn’t play nice with Net::CalDAVTalk.

Whether this is a problem with Nextcloud (which doesn’t try to do anything on its own – it uses SaberDAV underneath) or this is a problem with Net::CalDAVTalk isn’t a terribly fruitful pursuit. What is almost trivial to do is to ask Perplexity AI to generate the Perl program without Net::CalDAVTalk. The whole thing can be done with HTTP::Request, LWP::UserAgent, and XML::LibXML.

It is astonishing how well this is working, and how quickly this change is taking place.

Let’s Encrypt for my internal domain

It is time to renew my wildcard SSL certificate for an internal domain I have, and here are the steps I went through to solve it. When I say internal domain I’m referring to a DNS domain that exists on the public Internet, but which wholly and only points to the IP address of my home broadband router. That router has pass-through enabled, so that essentially, my pfSense box is my presence on the Internet for everything inside my home.

I turned off HAProxy so that pfSense wouldn’t be sending the challenge traffic to the only internal server I put out there. The internal server, Nextcloud, doesn’t play nice with others; in order to keep things consistent, they want it to be an appliance where the only stuff running on the box is their code. Okay, I get that. This wouldn’t be so annoying if it wasn’t bug-riddled junk that is in a huge rush to implement new features. Can you say “AI”? But I digress.

I created a new Linode API key in case the problem was that the old API key didn’t have access. Well, the first new key had the wrong selector, and resulted in “Your OAuth token is not authorized to use this endpoint”.

The problem is that the pfSense script is trying to generate a challenge key and insert it into a web server that doesn’t exist. The pfSense web admin portal is that web server. When I turned off HAProxy, that should have opened it up. It did, but I couldn’t tell because the Linode API key was wrong.

Okay, maybe I need to log in to the pfSense box and manually use a generated challenge key? How to log in to the pfSense box? When was the last time I did that?

Here’s a convenient command:

 history | awk '{$1="";print substr($0,2)}' | grep "ssh " | grep -v history | sort | uniq

We run the output of the history command through awk to remove line numbers, then search for "ssh " (the trailing space omits ssh-copy-id and such), run that through sort, and run that through uniq. Et voilà, and I have a list of all twelve boxes I’ve logged in to since history.

Sigh: pfSense isn’t one of them.

But this was a good exercise: I did get logged into pfSense, and did find the “Your OAuth token is not authorized to use this endpoint” problem.

I deleted the previous Linode v4 API certificate specifications, and it worked.

Time to turn HAProxy back on.

Okay, the short form is:

Generated a new Linode API access token with Domain read/write access
- This probably won’t be required if the access token hasn’t expired.
pfSense > Services > HAProxy > Settings > disable and apply
pfSense > Services > Acme > Certificates > pick certificate and Edit > delete the Domain SAN list entry > Add a new Domain SAN list entry with the new Linode API access token > Save
pfSense > Services > Acme > Certificates > pick certificate and hit Renew
Do the other certificate in the list
pfSense > Services > HAProxy > Settings > Enable and apply

Nextcloud has far too many bugs

I like Nextcloud: it is probably my favorite piece of software that I run. But man it has a ton of bugs. Their support forums are full of people reporting problems, and there is no solution.

Sometimes, the support forums do report that “Yes, this bug is listed in the Github bug tracker.”

There are more than 2,300 open bugs.

It is ridiculous. I saw a changelog that said the update would fix a bug I was seeing. I installed the update. The bug was still there. Quality control in this project is deplorable. I have another bug that I experience daily that has been open for almost two years.

What is dismaying to me is that the main developers have a Microsoft mentality: let’s add new features! No, we’re not going to work on bug fixes: debugging is boring.

In a couple of years, after I retire, I may decide to learn PHP programming. I haven’t really been fond of PHP.

Way back when, I read (well, got through the first few chapters) of a programming book that pointed out that software can be written to be mathematically provably correct. For every memory allocation, the math can add to the sum of debt. Memory should be specifically de-allocated, which subtracts from the sum of debt. When at the end of your source code, the sum is zero, you’ve handled all cases of allocating and de-allocating.

Nothing about PHP makes me think this is true for that language. Maybe I just don’t know the language well enough.

But, PHP does run a ton of super successful projects. So there must be something there which is valuable.

But yeah, I’m not going to be competent at writing PHP for years. Hopefully, someone at Nextcloud will get tasked with fixing bugs before then.

Temporary fix for Nextcloud calendar broken sync

Nextcloud has a nice home page called the Dashboard, which has calendar items and to-do list on it. But ever since Calendar App version 4.5, it has been broken for items sourced outside of Nextcloud. In other words, if you create a calendar item on your smartphone and sync it in to Nextcloud, on the Calendar web page you can see the item, but on the Dashboard home page it will be missing. The solution is to downgrade the Calendar app to version 4.4.5

Steps to perform:

In the Nextcloud admin interface, find the Calendar app and disable it
ssh into your Nextcloud instance
cd /var/www/html/nextcloud/apps/
mv calendar calendar-old
wget -q https://github.com/nextcloud-releases/calendar/releases/download/v4.4.5/calendar-v4.4.5.tar.gz
tar xvf calendar-v4.4.5.tar.gz
chown -R user:group calendar
In the Nextcloud admin interface, select the Disabled apps section. Then Enable (but not update) the Calendar 4.4.5 app.

And now, when you go back to your Dashboard home page, your calendar will have all the items on it. 🙂

You do get to apply this fix after every update. 🙁

Technically, this post title is somewhat misleading: sync is not broken. What is broken is that items that sync in from CalDAV sources apparently have something that, when it is present, causes the Dashboard page to skip those calendar items. It just looks like sync is broken because you knew the items were on your calendar: but when you look at the Dashboard for today, they are missing. I suppose a better title would be Temporary fix for Nextcloud calendar (some) items missing from Dashboard

Ogg > MP3 (thanks, Apple) (not)

I have several CDs (Compact Discs, not Certificates of Deposit) of music that I like. When I popped them into my PC, I got several folders of files I could copy from. I chose to copy the .ogg files because I liked the idea of using an encoding format without weird licensing issues.

Apple has foiled that plan. If I try to play a playlist on an Apple device, the .ogg files get skipped because (apparently) Apple doesn’t feel like playing nice with the Open Source community. They may have more money than God, but adding another codec – that doesn’t have license issues – to their devices isn’t something they are going to spend money on.

When I work on-premises in the office, my co-workers are often noisy and annoying. I want to pop in my Airpods and play background music to drown out their inane chatter. I don’t want to carry the music files on my device; but I do have a Nextcloud server at home that can stream the audio from the Music app web page. I can log in on my iPhone and play the playlist.

But because it’s an iPhone, it auto-skips the Ogg Vorbis files. This doesn’t happen when I’m at home playing the same playlists on Linux or Windows.

So now I get to re-copy the files from the physical media to my NAS (network attached storage) which in this case is a Synology.

First, I get to delete the files with the .ogg file extension. Two steps (for example):

exiftool -p '$filename' -if '$album =~ /WOW Worship: Yellow $disc 1$/' *.ogg > wow_worship_ogg_file_list
This generates a file, wow_worship_ogg_file_list, which has the file names in a list.

then to delete them:

xargs -I{} rm -r "{}" < /path/wow_worship_ogg_file_list

Second, after having cleared out the disk space, I can copy from my physical CD to my NAS. That takes a while; and, after it is done, the file names aren’t wonderful. Rename music files to their title to the rescue.

Except, of course, for a duplicate file name. I have an MP3 file I bought from Amazon (published by Monstercat) with the same title as one of the files from the WOW Worship CD. I would prefer to rename the Monstercat file, but really if I’m going to be running the rename music files to their title command often, I need to change the Title inside the .mp3 file. If I don’t, the next time I run it, it will attempt to rename the file to a duplicate name that is already in use.

Exiftool doesn’t really write new Titles, apparently. I think it can, depending on the file type. I wonder if the weird license problems of MP3 are at the root of the problem. Whatever: the answer was to add the id3v2 program and use it instead.

id3v2 -t 'Title by Artist' file.mp3

Now the rename music files to their title script moves the one file to the new file name, and the other file to its simpler file name than what came off the CD.

New Nextcloud setup with cron and transactional file locking problems (solved)

I set up Nextcloud on a new instance of Debian, and thought I had added all the pieces for memory cache and file cache, and had set up cron to run php -f /var/www/html/nextcloud/cron.php correctly. But in the Administration Overview screen I was still seeing this:

Last background job execution ran 2 hours ago. Something seems wrong.
The database is used for transactional file locking. To enhance performance, please configure memcache, if available.

But had installed Redis and APCu and configured them … so what is wrong?

I should mention that I’m using php 8.2. Apparently, with that new version of php, the APCu code now needs an additional setting that wasn’t needed before.

Find your way to /etc/php/8.2/mods-available and edit the apcu.ini file. Add this:

apc.enable_cli=1

Finally! I have the green check mark: All checks passed.

How to test if you cron job is going to run correctly:

sudo -u www-data php -f /var/www/html/nextcloud/cron.php

I had to add the sudo package to Debian, because the basic server build did not come with that. But what it does do, is let me switch user and do the command. First, I specify the same user that Apache is going to use: www-data and then I run the PHP interpreter, using the file /var/www/html/nextcloud/cron.php

Prior to the change, it erred out with a rather ugly OCP\HintException: [0]: Memcache \OC\Memcache\APCu not available for local cache (Is the matching PHP module installed and enabled?)

Now after the change it simply runs without reporting anything (everything ran sucessful)

Nextcloud web on mobile: remove iOS App prompt

New update: A little while ago, I updated my Nextcloud client on my iPhone, and a feature has been added to the “Use The App!” pop-up to turn it off. Thank you! I no longer need to mess with layout.user.php file on the server.

insert into oc_appconfig(appid,configkey,configvalue) values ('theming','iTunesAppId','');

There is a file in Nextcloud, layout.user.php which pushes a link to the iOS app. I wish to remove that. It will come back, every time Nextcloud pushes an update, so here is a note on how to remove it.

The file is (web root)/core/templates/layout.user.php

The three lines to delete are:

<?php if ($theme->getiTunesAppId() !== '') { ?> <meta name="apple-itunes-app" content="app-id=<?php p($theme->getiTunesAppId()); ?>"> <?php } ?>

I don’t know if there is a setting anywhere in Nextcloud which would let me simply disable the prompt to use the app. I don’t see such; so the fallback is to edit the source code. It would be nice if there were a setting stored in the database instead. When the Nextcloud people publish an update, the recorded setting could keep the “Use The App!” prompt disabled. But I don’t think there is a setting I can get at which lets me control this.

That php code implies that the theme could have a setting; but I don’t see anything in the theme setting page for a flag of getiTunesAppId.

Mostly, I use the other pieces of Nextcloud: Calendar, Tasks, Music, and Dashboard. For those, that “Use The App!” prompt is a waste of screen real estate on a platform that doesn’t have a lot of screen real estate to waste.

I wouldn’t terribly mind if the “Use The App!” prompt showed up if I were to visit the Files part of the Nextcloud web UI: that’s the only thing the iOS app is good for. But I almost never use the Files part of Nextcloud when on mobile.

I’ve asked the people in the Nextcloud support forum if I’m missing something; this could be a solved problem already. If I get an answer back, I’ll update this post.

User dakkar-tn did answer my post with the database query above. I am very appreciative of his help.

exiftool to playlist file

Assuming that you are currently in the subdirectory with music files, and those files are of type .ogg and you want to create a playlist file named _great.m3u which contains the names of the songs with “World of Warcraft” in the album name, this one liner creates such a file:

exiftool -p '$filename' -if '$album =~ /World of Warcraft/' *.ogg 2> /dev/null > _great.m3u

Assuming that you then wanted to add the files of type mp3 from the artist E.S. Posthumus, this one liner adds to that file:

exiftool -p '$filename' -if '$artist =~ /E.S. Posthumus/' *.mp3 2> /dev/null >> _great.m3u

Assuming that you then wanted to add files of type mp3 with the Genre of “Classical”, this one liner adds these to that file:

exiftool -p '$filename' -if '$genre =~ /Classical/' *.mp3 2> /dev/null >> _great.m3u

Assuming that you then wanted to add files of type mp3 with the Comment of “VIVA EL PRESIDENTE!”, this one liner adds these to that file:

exiftool -p '$filename' -if '$comment =~ /VIVA EL PRESIDENTE!/' *.mp3 2> /dev/null >> _great.m3u

exiftool is the wonderful utility written and maintained by Phil Harvey

-p '$filename' prints the file name. We later strip off the other stuff by redirecting stderr to null. That’s the 2> /dev/null part.

-if '$album =~ /World of Warcraft/' and -if '$artist =~ /E.S. Posthumus/' are matches against a regular expression. =~ says we are doing a match and the text between the slashes are what need to be present for the match to report true.

> _great.m3u overwrites the existing file, but then >> _great.m3u appends to it.

Care to guess who purchased the collector’s editions of some of the games so I could get a CD of the game music (or files from Steam)?

One thing (I don’t know that it’s a problem, really) is that Artist =~ /E.S. Posthumus/ will find the same file as Genre =~ /Classical/ so the same songs end up in the playlist twice. Maybe I just like E.S. Posthumus so much that I want their chance of being picked by the shuffler better than average. 😉

But if that’s not your bag, this will make a new file (with a new name) which contains only unique song file names:

sort _great.m3u --unique --output=_great-unique.m3u

If you happen to have your own Nextcloud with the Music Player app, you can import this _great-unique.m3u file directly into a new playlist.

The Helm migration is complete

As I mentioned before, The Helm email appliance company is calling it quits, which I understand. If the business isn’t going to make it, it is better to pull the plug than just keep letting things linger. Best of luck to them on their next adventure.

So, what did I do?

(there was a detour while Amazon pissed on their customers wanting to run Mail-In-A-Box) (me)
I provisioned the smallest Ubuntu 22.04 LTS machine that Linode has.
- Mildly annoyed that it doesn’t really support LVM (Logical Volume Manager); they have a backup service that runs an agent inside their machines, and that agent doesn’t do LVM. Still, I know that I’m going to need to grow disks, so I had to learn how to re-partition the Linode so I could do LVM. LVM done.
I made a mail server on the Linode machine at a domain name I have that I don’t really use. I followed the excellent guide from Christoph Haas at workaround.org: ISPmail guide for Debian 11 “Bullseye”
I got RoundCube webmail working for the domain name; complete with SPF and DKIM.
I got Thunderbird to send and receive from the domain name.
Then I added Nextcloud to the same box. I wanted CalDav for contacts and calendar, when I eventually hook my iPhone to it.
- The Nextcloud documentation really needs a lot of work here. If I were retired, I would like to help them with their documentation.
- Finally, I have the files.example.tld function of The Helm replaced, although at a different domain name.
- Rspamd uses Redis, but so does Nextcloud. But one uses the network stack, and the other, Unix sockets. Get them both set same.
Then I added Duplicati backup. This wasn’t great, as it added a ton of overhead in the form of Mono, just for a graphical user interface.
I realize that I’m going to want to host my WordPress here too. I don’t want to have to wrangle four Let’s Encrypt SSL certificates, one for each domain. What about a single wildcard SSL certificate?
- Yes, that can be done, but: my domain names registrar doesn’t support it. Linode does, though. I install the Linode DNS agent on my machine, and spin up Linode DNS servers to do the DNS work. I have to configure my domain names registrar to tell the rest of the world that Linode is where my name servers are.
- Somewhere in there I installed the Unbounded DNS resolver. Looks like I need this on my home machine, too, for Home Assistant.io¹
I got to the point where I could request the domain name transfer. Turns out the people at The Helm were going through Ghandi.net. Ghandi.net tooks as long as they legally could, before actually doing the DNS transfer.
- Ghandi –> registrar, then registrar to point to Linode. Linode DNS needs to be reconfigured for SPF and DKIM. I had gotten some DNS records wrong, too.
Thunderbird to connect to the mail.domain.tld, and though the name hasn’t changed, everything underneath has. Thunderbird is not happy; I lose all my old mail.
- Well, I didn’t, but it is in a new folder now, so that I’ve got an old version of my mailbox and a new version of my mailbox, and they are separate. Not ideal. Perhaps I could have done an IMAP to IMAP transfer, if I hadn’t already moved the domain name.
Hey, looky there: one of the volumes filled up (but everything else was unaffected). Time to grow a disk using LVM.
iPhone to connect to CalDAV; phew that was not well documented and had tons of conflicting information.
Not really happy with Duplicati, so I remove it and Mono, and install Restic backup instead.
Okay, so the last thing left to do is to migrate this blog from Amazon to this new Linode machine. The transfer using NS Cloner goes well, as it usually does. But domain names need to be updated via Let’s Encrypt certbot.
- Crud. I’m on holiday out of town with family, and have only a Windows laptop with me. Per best practice security protocols, I can only ssh in from home. Logging in via root@ is blocked, and I don’t think I can even do a ssh-copy-id without getting in first and lowering the root login barrier. The certbot to add gerisch.org to the domains list is going to have to wait.
Here I am, at home, and I’m done. Dovecot, Postfix, RoundCube, Nextcloud, and WordPress all on one box.
While I was on holiday, I took the .mp3 files on the Nextcloud, and made Nextcloud Music Player playlists for the different types of files. Then on the 16 hour drive home, my iPhone logged in to the Nextcloud web interface and played playlists.
- It’s a bit of nirvana to me, to have a large list of songs (randomized of course) playing absolutely advertising-free because I paid for the songs in the first place.

I ended up not connecting Home Assistant to their cloud ↩︎