YOURLS really is awesome

For a long time, I’ve used YOURLS in my URL shortener projects. I have always liked extending it, so that it did something more than just URL shortening. The results of my work have turned, over the past two years, into what tny.im is today.

Until a few days ago, tny.im was running with software based on YOURLS. Yes, “based on YOURLS”. It wasn’t running “on top of” YOURLS. As I added more features to this URL shortener, I found it easier to just modify the core files and add rows to database tables at will. This came with a price: updating YOURLS without losing my modifications and while keeping database compatibility was really hard, requiring me to rewrite all the modifications. To make things even worse, I had modified some of the core files for them to work with the Bootstrap CSS and JS things. The statistics page (yourls-infos.php), which I had also managed to modify to a point where not only it was Bootstrap-themed, but was also the main UI for users to edit short URLs, was a specially hard problem to solve if I was updating YOURLS.

I could have stayed running the 1.5 version of the said script forever. However, it lacked many under-the-hood improvements of the 1.6 versions, and as new versions would be released, it would only become more obsolete. Again, a big problem resided on the link statistics page: it wouldn’t handle links with many clicks properly, because it would use a lot of memory. This bug no longer existed on 1.6, and this was something that kept me thinking I really should update to a newer version.

Another thing that I had heavily modified for tny.im was the public API. I had modified it to not support some methods which would disclose too much information (e.g. long URLs for paid-access links and links which had reached their hit limit), as well as added support for tny.im-specific features like the passcode, hitlimit and Bitcoin related things.

My code for all these things was really ugly. It had been progressively added and changed over the course of two years, had multiple coding styles, multiple indentation styles, multiple bug styles and God-knows-what-else. The fact that I only worked on the project sporadically meant that I often didn’t remember what I had done already, meaning there were giant mistakes like two variables for the same thing. Somehow it all worked, well enough to take to the correct destination over two hundred thousand short URL clicks.

Some days ago, I finally decided it was time to do something. I wanted to add new features to tny.im and future-proof it at the same time, but the code was impossible to maintain – the original core code, mixed with my bad code, made it seem it would inevitably break if I touched it. I knew, from the start, that the proper way to add features to YOURLS was to code plugins – but laziness, convenience and the fact that version 1.5 didn’t offer that many plugin hooks made me modify the core files, as I said above.

I had to start from scratch, doing things in a “staging” vhost that had no communication to the live tny.im website. I started by installing YOURLS on that vhost, on a database separate from the live one, of course. Then I accepted the challenge of trying to implement all the tny.im features, including the Bootstrap theme, without ever touching the core files of YOURLS.

Fortunately, version 1.6 of YOURLS had many more plugin hooks. But I knew I wouldn’t be able to implement every feature as a plugin. Things ended up like this: I consolidated my hackish plugin soup from the old tny.im scripts in a single “tnyim-framework” plugin, and things like the index page UI, link lists/folders, Bitcoin address shortening and “internet toll” features, as well as login/logout (which is separate, and has always been, from YOURLS auth methods) would keep being separate from YOURLS.

Instead of modifying core files like functions.php and functions-html.php to add my code, I put them in a separate tny.im-specific folder. It has it’s own “load-tnyim.php” file, in the style of the “load-yourls.php” file, which loads the necessary variables and files for the tny.im features and UI.

The problem with the statistics page, as well as some modifications I had made on yourls-loader.php, was solved my creating tny.im-specific files that are similar to the YOURLS ones and perform the same functions, but have my modifications. The new statistics page is based on the YOURLS 1.6 one, and it was a hassle to modify to meet the functionality of the old one, that is, Bootstrap-themed, and with that “Manage” tab that allows people to edit links. This was mainly because, as @ozh says, it is an “awful HTML/PHP soup”. Also, it has little to none plugin hooks from what I can see, but even if it had, they would never be enough to allow me to customize it to the point where I did.

Things like link preview, and the API modifications, went in the tnyim-framework plugin. And the reason why I think YOURLS is awesome, is that I managed to change much of the API behavior, adding new return and request fields, as well as obfuscating some, just with plugin hooks.

As for the database, rows specific to tny.im are now kept on their own table. The functions.php on the tny.im folder has methods for adding and editing URLs that handle the tny.im features. It works like this: the code calls my custom methods for adding/editing short links; then, things are added/edited on the normal YOURLS DB table using the core methods, and then my code takes care of the other table which stores things like link hit limit and price (for internet toll links), as well as adding the link to the users’ account when appropriate.

Speaking of user accounts, the data that relates users to the keywords they have access to, is kept also separate from YOURLS tables. Same with the lists of links feature.

Finally I moved the columns of each database table to their correct destination in the new tables – five hours worth of SQL commands. Then I moved the new script files to the vhost of the live site, after editing the YOURLS configuration file and the nginx vhost config of course. It seemed to start working right away. All this took me two days.

Now the code for tny.im is much more clean, readable and most importantly, maintainable. I can finally add new features to tny.im without breaking half of the existing ones. And I can update YOURLS without breaking the whole thing, since none of the new features are implemented in the core files. @ozh’s improvements in pages like the links statistics one and yourls-loader will not automatically get merged with the tny.im code, but I can add them manually while keeping my changes.

I’m not yet fully sure the new tny.im is clear of bugs, but over 90% of it seems to work the same or better than before. It should look and work pretty much the same as it did before – which only proves, that I really made a wrong move when I started editing core files, since everything could have been done without touching them. I think I learned the lesson, hopefully not only for YOURLS, but for most scripts which have plugin interfaces.

I think I made the most full-featured URL shortener ever seen, and obviously I’m proud (certainly too proud) of it. It’s all built around YOURLS, and I bet you wouldn’t even tell it was powered by that script at a first glance. Stay tuned as more features and bug fixes are to come.

Utilities Beta 9 is out!

Six months after the previous public release of my Utilities add-in for the Casio Prizm calculators, the new Beta 9 is out! This version comes after an important refactoring of the source code, which by itself led to many fixed bugs and much higher stability. The g3a binary is also smaller (about 136 KiB) while featuring the same features plus some new ones, and that’s a big plus on a device with limited storage memory like the Prizm.

Changelog:

The most noticeable new “feature”, is probably the increased stability on tasks and calendar events, as well as in the file browser.

You no longer need to keep the storage memory fully optimized for calendar events and tasks to work without system errors.

Another new thing is a new timer/stopwatches system, which is now called “Chronometer”. There are 20 chronometers and each of them can go upwards or downwards. You can now select multiple chronometers (hint: Shift+F1 on the chronometers list shows useful selection options).

There’s now also a way to open files as text, with automatic line ending detection (supporting CR+LF, CR and LF) – on the file manager, press EXE when hovering on a file, then press F1 on the file information screen to open the text viewer. This still doesn’t automatically convert special characters from ASCII to Casio’s multibyte system but it’s something I’m planning to add.

File copying has been added, but this is unfortunately still too unstable, so this option will only appear when the “Show advanced tools” setting is enabled. Note that this is one of the last two functions of the add-in which are unstable (the other is the Add-In manager, which keeps being unstable probably because of a OS limitation – one can’t touch the add-in array under normal circumstances…).

Now there’s a way to save the current time to a OS Basic variable (those you can use on Run-Mat and almost everywhere in the official software): press the X/θ/T key on the main screen (where you see the time), and the hour fraction will be saved to the variable T with the maximum precision the RTC allows.

This add-in is finally headed for a stable 1.0 release – this Beta 9 release would be called a RC, if it weren’t for the Add-In manager and the file copying function, as well as a little feature that was committed to the git repo already in March, but got lost as I refactored the code. I shall add that little feature in time for the next release.

Update instructions:
(only applies if you used Utilities before Beta 9)

This bit is really, really important as there are some folder names that changed with this new release; also, not all files are compatible (calendar events and tasks are compatible, though). You must follow the following instructions for Utilities Beta 9 to work properly:

Delete the @UTILS folder in Main Memory (Memory app in Main Menu, press F1, scroll down to @UTILS, press F1, F6 and then F1);
– Now connect the calculator to your computer through USB, as a flash drive, and perform the following steps:
Rename the @CALNDAR folder in the storage memory of your calculator to @UTILS (you must do this on the computer because the OS won’t accept the @ as a valid character);
– Inside the newly renamed @UTILS storage memory folder, delete the file Hash.plp in case it exists (the new Hash.plp file is no longer compatible with the ones from previous versions).
– Copy the new utilties.g3a (download below) to the root folder of the calculator, overwriting the old one.
– Safely disconnect the calculator from the computer. You are ready to use the new version once your calculator finishes updating the Main Memory…

Download Latest Version of the Utilties add-in for the Casio Prizm