After a long hiatus (since I have been known to get distracted from time to time) I have continued working on Freeswitch with regards to OpenBSD. Before I get into the current state of affairs, I should mention that I do have I have a semi functional Freeswitch installation (from before FS started the -stable branch) running on a pair of OpenBSD 5.1-current ultrasparc VMs but the tears and pain of trying to keep that running and up to date was way too much for a mere mortal like myself to entertain. Moreover, the fact that my (unpublished) patches refused to work on a comparable 5.2 and subsequent 5.3 install told me that way lay dragons. The SSL fiasco and the new ressl work finally made me turn the systems off and resign them to the closet of dusty bits and reinstall on some linux boxes so at least I could have access to vendor patches.

Le Sigh.

Even to this day, the out of the box upstream install scripts don’t actually work on OpenBSD, and the software is absent from official OS repositories. Though I will nod to the fact that the upstream developers have made an attempt to get things working, they are doing stupid things in the process, like manually installing a magical ‘Freeswitch’ versions of openssl (stupid stupid stupid), curl and libedit. Honestly its like, once they cut their finger accidentally in the kitchen, and instead of cleaning and bandaging the injury, they went ahead and put their whole hand in a Garburator.

Anyway, I’ll try and not be too cynical here since nestled inside the monstrosity(oops sorry, still being cynical) that is Freeswitch are the necessary bytes to have a working SIP telecom platform. I’ve made some decent progress in getting the software to compile and not explode in my face, and even took the time to give the @ports mailing list some updates about project status.

• 2014-10-23
• 2014-10-31

Stuff that works (or at least says that it works)

• mod_callcenter
• mod_commands
• mod_conference
• mod_db
• mod_directory
• mod_dptools
• mod_esf
• mod_esl/mod_hash/fs_cli
• mod_fifo
• Lots of manual twiddling was required here, and I still haven’t turned on the c++ esl API  that lets swig bindings start working (Specifically the ESL bindings to stuff like lua, perl, Java, and C#).
• Moral of the story is that fs_cli works, I’m possibly going to re-write it or write my own from-scratch version at some point to be less terrible, though at least it uses getopt (but not the operating system provided version of getopt).
• By the way, fs_cli has no license?
• mod_loopback
• mod_sofia
• I hope this is the most challenging module to enable and have working correctly. In total I spent several days and many dozen cups of coffee at the local Greasy Spoon, banging my head against this module trying to get it to work. The module depends (obviously?) on the sofia sip library from Nokia and has been dramatically patched since the last official release. Considering the transfer of so much Nokia IP to Microsoft I would be willing to bet that many of these changes will never be pushed to the official Nokia upstream library (not that the FS devs pushed many changes even before the Nokia/MS thing happened). Also, I flirted a bit with the changes that have arrived in the 1.4 tree of Freeswitch, and don’t think much of them, (network layer violation badness).
• mod_others_ive_forgotten

I’ve been hand picking the modules that I want to turn on, and aggressively deleting anything that seems stupid to me. Anyone who eventually uses this port may find that their favourite module is not included with the port. Of course this is because my goals are different than upstream Freeswitch developers, this is will sometimes result in missing features and since their design, implementation or purpose are undesirable from my point of view.

Still to do

• Modules… many mods still need to be reviewed and enabled or deleted.
• Devise an appropriate default minimal configuration
• Fix hashing bugs (see my @ports mail)
• Launcher re-write (switch.c is full of badness… damn, cynical again)
• fs_cli re-write (how about a minimal level of security)
• String Slinging audit, there are lots of places I’ve seen that could be potential overflows.
• Memory pool draining; lets not see a repeat of stuff like heartbleed by not using memory pools.
• Update to 1.2-stable (current tree) as far as possible

A quick note on Project Goals

A reminder that this project is a ‘shallow’ fork of Freeswitch. I have no intention of going feature crazy and introducing anything other than a basic installation of Freeswitch that meets MY needs (SIP over TCP/UDP with TLS and SRTP). My goals specifically are feature and platform stability and security. I will be making attempts to pull security fixes from the upstream tree but I have no intention currently of ever updating past 1.2, if you think you need web sockets in your sip stack go install linux.

I have a customer which uses Windows SBS 2011 Essentials as a basic file and backup server. It runs on one of those little supermicro servers with an integrated atom chip and 4GB of RAM. Hardly a workhorse, but it is perfectly adequate for running his three person office.

Anyway, this morning I got a call saying that the server was down, and couldn’t seem to be restarted.

When this kind of thing happens, I usually presume that its some kind of minor user error (like its not plugged in, or the tooth fairies pulled out the network cable overnight) and almost always, I can get in and out within a few minutes and couple magical key strokes.

Well… Today was a genuine problem requiring some actual work.

Turns out the the hard drive (yes there is just one drive in the unit) suffered some sort of mechanical fault and started clicking rather badly.

Well, a quick run to the local computer shop and a few minutes of minor surgery yielded a perfectly functional, yet completely blank file server (being blank is very bad for a server). The value the server had nothing to do with the actual cost of the equipment, but its value rests entirely on its function as a repository of all the companies data and that data is on the dead drive.

Fortunately, I had seen this coming a long time ago and the machine has been dumping daily images to an old drobo sitting on the shelf for something on the order of 3 years. Btw, those old drobo’s aren’t good for much, but this is one job that they do pretty well.

Anyway, recovering the image was fairly straight forward (this server doesn’t have a DVD drive) so I needed to image a USB stick with the SBS 2011 Essentials installation disk, which works fine except for the fact that the Windows recovery software doesn’t like magical new drives showing up when its about to recover an image, so annoyingly I needed to disconnect the USB stick just after the software had identified the recovery image and before starting the recovery process otherwise I would be whacked with error 0x80070057 which basically says that the recovery system is shaped differently than it should be.

Yawn.

All in all the recovery process is fairly straight forward, and the only thing I’d like to see added is a way to clone the recovery images to an offsite storage thingy.

I find that I get distracted from the task at hand very easily.

My latest distraction from the FreeSWITCH on OpenBSD project has been a renewed interest in UltraSPARC hardware. I recently purchased a pair of Sun Fire V215 servers to act as routers, and on the way I got interested in the newer generations of UltraSPARC processors, which I think are very interesting for a variety of workloads that I have to deal with.

Anyway, this evening while out at my parents place I rummaged in the basement for a few minutes trying to find an old system I had left out here a few years ago.

Sure enough, sitting under a small pile of dust, I discovered my old Sun Blade 100 workstation. I had purchased the machine some years ago when I was working on a project which I wanted to ensure ran on big-endian architectures.

Plugging in the system, I was pleased to discover everything was exactly as I had left it (apparently in 2007 since it still had a OpenBSD 4.1 install disk inside it).

Quickly burning a copy of OpenBSD 5.3 for UltraSPARC, and running the install took about 10 minutes and I was pleased to discover a fully operational desktop.

OpenBSD 5.3 running out of the box on a Sun Blade 100 UltraSPARC workstation

These workstations were very useful in their day, and I am pleased to see that mine is still running.  Performance is a little lower than I would like for some activities like browsing the web, but its actually perfectly acceptable as a development environment (and nicely free of distractions too).

I’ve finally gotten around to making new headway with the FreeSWITCH on OpenBSD project. Although I can honestly see at least year (or more, really) of labour in front of me to produce a high quality port which I would be content to run my voice services on.

At the moment, I have a semi functional build which compiles a binary capable of executing just past the banner. I haven’t checked the last call trace yet, however it looks like we are breaking just after the process forks and spins up a bunch of threads.

Recent Work

I merged a commit today that replaces the hash table wrapper that used to point at internal sqlite APIs to a the public hash table implemented by APR.  This is in reaction to an instant crash bug that I was encountering when the old system tried to access the internal sqlite malloc code.

I’ll note that the original implementation of the hash table stuff used APR in the good old days (2008) and was replaced by Anthony for reasons that I am currently unclear about.

The results tested well in an impromptu set of test cases I whipped up to see if things should work, and the configurations loaded successfully, however the test of the new implementation really will only happen down the road when we start flinging calls around. 

Observations

I find the code style of the FreeSWITCH developers difficult to read for a couple reasons:

• Long symbol that_describe_what_should_be_happening_like_this
• Long prototypes that disappear off the edge of my monitor, I’ve seen lines with more than 180 characters which is a bit much.

I’m not going to whinge about the code style too much since it is what it is, however I’ve voiced my official note saying I don’t like it.

Another issue I have is with the munging of dependencies. What I mean here is that the developers have taken a number of liberties in changing the internals of some dependencies. I suppose that they had their reasons, however it does make me twitch.

The more I fiddle with the internals of FreeSWITCH, the more I approve of the projects that have come from the OpenBSD guys. Especially when it comes from the perspective of portability, where the dependency trees are vanishingly small.  The FreeSWITCH perspective is somewhat along the lines of “ok, this pile of kitchen sinks seems balanced, don’t fuck with it”.

Current tasks

There are couple items to do right away. The first is spin up the module loading code, port some modules, and verify that the core modules are loading in what appears to be a correct manner.

Also I noticed that I am statically linking my internal libfreeswitch library and that needs to stop.

Down the road

I really dislike the manner in which freeswitch loads its configuration. On the horizon is a new launching and configuration loading change that makes me less twitchy. I’ll make sure that everything I do here could be conceivably imported upstream.

Also, there are couple areas that I think may need special attention, memory handling and threading

Summary

There is lots left to do!

I’ve been a long time user of OpenBSD for various projects since I worked for Nortel back in the good old days (2004? Wow, that’s nine years and counting). My experience with the operating system is mostly on howling servers that you would never run on your desk.

Anyway, I’ve been working on porting some software to OpenBSD for a couple months and kept thinking to myself

“Boy I should get a proper workstation setup”

At first I was just using an ssh shell to one of my servers and working off that, which was a pain in the ass when the network was slow or out, or otherwise unavailable.

Second thought was,

“Maybe a VM on my main laptop will do?”

Uh, no. I don’t like VMs, and to be honest I need a bit more performance than typing into a VM console. I have friend that is a VM hipster who thinks they are the best thing since butter, but I’m just not that kind of guy.

Anyway, after setting up a workstation on a spare P4 that has been sitting on the shelf behind me for a couple years and enjoying the experience, I grabbed a Dell D620 from a local company which sold all its old gear to me for a tuppence.

Power Management

Last year I sat through a talk by Theo de Raadt where he claimed that OpenBSD has the best ACPI implementation of any of the free operating systems. After noticing that the stock installation of OpenBSD ran a little hot on the D620, I opened the acpi(4) man page which said:

“Userland may access acpi by using the apm(4) device.”

After a little bit of reading, I concluded that editing rc.conf.local to start the apmd daemon (responsible for handling the power management stuff) in “keep my computer cool mode” should do the trick.

apmd_flags=”-C”

On the advice of a random webpage I un-commented the line in /etc/sysctl.conf which tells the system to suspend whenever the laptop lid is closed.

machdep.lidsuspend=1

After a nice demo by Henning Braur after his talk at BSDCan this year I am pleased to confirm a nice suspend and resume experience. Not only does the laptop take a nice nap when the lid is closed, it actually gets out of bed and back to work when its open.

Docking

Along with the laptop came a dock, which I specifically wanted to enable a modest transition from mobile to fixed workstation. And after a solid 7 minutes of rummaging through man pages I couldn’t find any hooks to detecting when the machine has been docked.

Basically I wanted the following to automatically happen when the laptop is docked

1. Turn off wifi
2. Disable suspend on closed lid
3. Turn on my desktop monitor
4. Turn off the LVDS laptop panel

1 and 2 require some root permissions which means configuring sudo, 3 and 4 are easy with xrandr (thanks keith packard).

Upon undocking, I want to reverse this behaviour.

So far I have two scripts sitting on my desktop called docked and undocked which I can run to manually cut things over to the preferred state. I suppose this will do for now, since I have other things that need my attention.

Does anyone know of a more elegant way to do this? Or would someone like to write up a small daemon to handle this in the background? Maybe it could be called dockd?

Summary

In anycase, getting the system up and running has not been terribly difficult, and I should have no more excuses when it comes to getting to work on the OpenBSD port of FreeSWITCH I was going on about recently.