Projects, Technology 2017-09-04

OspreyForth Update, September 3

This post is a quick update and summary of my project thus far. Things have been progressing reasonably well, but there are always hurdles, so one can only hope for the best.

The Current Status

I’ve run in to a bit of a showstopper. The system compiles, but running it yields a segmentation fault. This is a classic case of not doing due diligence. Several years ago, I created the first version of OspreyForth on a PowerPC Powerbook. It compiled fine and I could do basic Objective-C calls with it. Based on this, I assumed (don’t do that) that there would be nothing wrong with the base FICL interpreter I was using. As it turns out, the segmentation fault is occurring at some point while the virtual machine loads, which places the point of failure directly onto the FICL interpreter and not any code I’ve generated myself.

The current solution seems to be to connect with the FICL maintainers and let them know of this issue. Something must have changed with how OS X handles things to cause a conflict. Because there is no Mac-specific makefile, I would guess that there are no Mac users in the group of maintainers. I won’t know until I contact them, however.

The Current Story

When I started OspreyForth v1 and v2, I did so with the desire to create a Forth that could take advantage of the Objective-C Runtime on OS X and GNUStep. I know others exist to one degree or another that can do this, but I wanted something especially for myself and geared toward me in particular. I believe it was Forth’s creator, Charles Moore, who advocated the idea that everyone should make their own Forth. Such a Forth would then be geared towards the specific needs of the person who made it. I was kind of following that philosophy.

Having said that, my circumstances changed. There were certain issues with my home Internet connection that was making things problematic. I had recently upgrade my web hosting solution to a VPS, so I had plenty of space to play with if I could just get OspreyForth to run off my site. My hosting provider is Dreamhost, however, and in order to run GNUStep on their Ubuntu servers, I’d have to sign up for their cloud computing solution. I’m still undecided about that.

Basically, GNUStep was out and my home connection was too problematic to use, so going with an Objective-C connection seemed no longer worthwhile. I decided to go with a plain C connection and set about getting the libFFI libraries and headers. It was a little time-consuming, but the documentation was easy to understand. I incorporated the FFI library and incorporated the dynamic library functions from Unix’s dlfcn.h header. I figure if I ever need to run OspreyForth on Windows, I’ll use the Cygwin library to build from.

Where to Go From Here

At this point I need to see if this segmentation fault is resolvable. I’ll contact the people behind FICL and hope to high heaven that this is an easy to fix issue. If not, I have to weigh my options. If it becomes a lost cause, I’ll move over to GForth. I really am loathe to do so, however, because I don’t like how GForth handles foreign function interfacing. Basically, I would create some Forth code that then gets translated to C, which then gets compiled to a dynamic library, that then gets put somewhere in the OS. This will almost certainly run afoul of Dreamhost’s policies (I already can’t access anything that requires sudo, for example). Since I will almost certainly need access to sockets and other resources when I’m done, this is a problem. I could, if I needed to, get GForth to talk to some Perl or Python script that has access to the resources I would need, but this seems like a very sub-optimal solution. GForth would almost certainly be an act of last resort, if at all. If nothing works, I may just move over to Perl or something like that.

Still, I’ve spent enough time on this I would like to see it through. I have my fingers crossed for a FICL solution!

Update September 5, 2017

After writing the above post, I decided to go back to playing with pForth for a little bit. One of the reasons I abandoned that interpreter was because I couldn’t reliably get C Strings out of the system. This meant I couldn’t call outside libraries or functions. It turns out the reason for this was because I was trying to use standard strings (for example something like s" Hello", which would yield some address for the string and a separate number indicating the number of characters in that string. What I should have been using was counted strings which in which there is a byte indicating the number of characters in a string followed by the string (this is all at the same memory address).

This is awkward for a number of reasons. First, counted strings are limited to 255 characters by their very nature. Second, the pForth API call that converts the counted strings to C strings also requests a separate number indicating the letter count. That means I have to make the counted string, call a Forth word (function) to count the number of characters in the string and then submit both. Keep in mind, counted strings already encode the number of characters in the front of the string. So, overall, this doesn’t make a lot of sense.

I think what I may have to try in the future is creating a regular Forth string, using memcpy to copy the string over to a newer and larger string and then add the NULL character to the end to make it an official C string. As it is, though, I have what I need to make API calls and load dynamic libraries. I just can’t deliver large paragraphs. Worst comes to worst, I’ll make a bunch of counted strings, convert them to C strings and then use strcat to concatenate them together. A bit hackneyed, but we’ll see.

Update September 6, 2017

Well, I’m stumped. Last night on a lark I decided to try the memcpy idea I mentioned above and it worked on the first try! I don’t know why I was having such problems with string conversion before. Now, I haven’t tried converting the C strings back into Forth strings, but I do not seriously think I will need Forth strings a lot beyond creating text to pass on to outside libraries. Most of the string manipulation facilities I have available to me are in C-based libraries anyway. The only thing I have to be careful about is the creation and destruction of the strings since malloc is involved. So I’ll have to malloc the final string and then free it when I don’t need it any more. If I can keep track of that, I think I’ll be OK. I’ll also need to add in dlfcn and LibFFI.

I guess this means it’s back to pForth.

About the author

Erik Hentell: I started out in theater before moving to graphic design. I eventually moved into web design while trying to expand my knowledge on software development. I currently work for a media company helping with their digital assets such as source code archives and ebooks.