Hardware-software compatibility on handhelds

It would be good if one could go to a store, buy any smartphone or a tablet, and later choose what OS to run on it. Like what happens with desktop and laptop computers, you buy any one of any brand (even Apple!) and you can almost certainly run a OS other than the default on it – some things are taken for granted on a PC, which allows for this kind of freedom of choice.

Sure, you can install custom ROMs on smartphones… now go ahead and buy any one that isn’t popular and see what happens. If there is not a big enough user base most likely there isn’t anyone to develop custom ROMs. And unlike what happens with PCs, there doesn’t seem to be some list of assured features and standards. On personal computers, even on the ones built from OEM parts and no recognizable brand, it’s as simple as setting a boot device that is an OS installation disk. Sure, you need drivers to get the full potential of the hardware. Still, some things Just Work(tm), like PS/2 keyboard and mouse, serial ports, VGA video out (and I don’t mean graphics acceleration) and disk I/O.

Isn’t it about time some sort of standardization starts happening for mobile devices too, both at the hardware (charging connector/method) and software (system booting, basic elements)? The idea would be that simple things like storage I/O, screen (without GPU), touchscreen input (even in single-touch mode) and hardware keys Just Work(tm) because they are really easy to drive by the OS?

Look at something like KolibriOS. It is pretty limited on what it can do on a PC, because of a limited amount of drivers are included, and anyway everything is very simple. But as things are now, there’s no way one could write a KolibriOS that would run on all the phones and tablets the same way Kolibri runs on almost every PC, even with limited functionality.

It’s certainly a distant dream, seeing that even charging connectors are something we are yet to get quite right… all to make big brands happier at the cost of unhappier customers – even if they never noticed they were less happy because of that. </rant>

YOURLS really is awesome

For a long time, I’ve used YOURLS in my URL shortener projects. I have always liked extending it, so that it did something more than just URL shortening. The results of my work have turned, over the past two years, into what tny.im is today.

Until a few days ago, tny.im was running with software based on YOURLS. Yes, “based on YOURLS”. It wasn’t running “on top of” YOURLS. As I added more features to this URL shortener, I found it easier to just modify the core files and add rows to database tables at will. This came with a price: updating YOURLS without losing my modifications and while keeping database compatibility was really hard, requiring me to rewrite all the modifications. To make things even worse, I had modified some of the core files for them to work with the Bootstrap CSS and JS things. The statistics page (yourls-infos.php), which I had also managed to modify to a point where not only it was Bootstrap-themed, but was also the main UI for users to edit short URLs, was a specially hard problem to solve if I was updating YOURLS.

I could have stayed running the 1.5 version of the said script forever. However, it lacked many under-the-hood improvements of the 1.6 versions, and as new versions would be released, it would only become more obsolete. Again, a big problem resided on the link statistics page: it wouldn’t handle links with many clicks properly, because it would use a lot of memory. This bug no longer existed on 1.6, and this was something that kept me thinking I really should update to a newer version.

Another thing that I had heavily modified for tny.im was the public API. I had modified it to not support some methods which would disclose too much information (e.g. long URLs for paid-access links and links which had reached their hit limit), as well as added support for tny.im-specific features like the passcode, hitlimit and Bitcoin related things.

My code for all these things was really ugly. It had been progressively added and changed over the course of two years, had multiple coding styles, multiple indentation styles, multiple bug styles and God-knows-what-else. The fact that I only worked on the project sporadically meant that I often didn’t remember what I had done already, meaning there were giant mistakes like two variables for the same thing. Somehow it all worked, well enough to take to the correct destination over two hundred thousand short URL clicks.

Some days ago, I finally decided it was time to do something. I wanted to add new features to tny.im and future-proof it at the same time, but the code was impossible to maintain – the original core code, mixed with my bad code, made it seem it would inevitably break if I touched it. I knew, from the start, that the proper way to add features to YOURLS was to code plugins – but laziness, convenience and the fact that version 1.5 didn’t offer that many plugin hooks made me modify the core files, as I said above.

I had to start from scratch, doing things in a “staging” vhost that had no communication to the live tny.im website. I started by installing YOURLS on that vhost, on a database separate from the live one, of course. Then I accepted the challenge of trying to implement all the tny.im features, including the Bootstrap theme, without ever touching the core files of YOURLS.

Fortunately, version 1.6 of YOURLS had many more plugin hooks. But I knew I wouldn’t be able to implement every feature as a plugin. Things ended up like this: I consolidated my hackish plugin soup from the old tny.im scripts in a single “tnyim-framework” plugin, and things like the index page UI, link lists/folders, Bitcoin address shortening and “internet toll” features, as well as login/logout (which is separate, and has always been, from YOURLS auth methods) would keep being separate from YOURLS.

Instead of modifying core files like functions.php and functions-html.php to add my code, I put them in a separate tny.im-specific folder. It has it’s own “load-tnyim.php” file, in the style of the “load-yourls.php” file, which loads the necessary variables and files for the tny.im features and UI.

The problem with the statistics page, as well as some modifications I had made on yourls-loader.php, was solved my creating tny.im-specific files that are similar to the YOURLS ones and perform the same functions, but have my modifications. The new statistics page is based on the YOURLS 1.6 one, and it was a hassle to modify to meet the functionality of the old one, that is, Bootstrap-themed, and with that “Manage” tab that allows people to edit links. This was mainly because, as @ozh says, it is an “awful HTML/PHP soup”. Also, it has little to none plugin hooks from what I can see, but even if it had, they would never be enough to allow me to customize it to the point where I did.

Things like link preview, and the API modifications, went in the tnyim-framework plugin. And the reason why I think YOURLS is awesome, is that I managed to change much of the API behavior, adding new return and request fields, as well as obfuscating some, just with plugin hooks.

As for the database, rows specific to tny.im are now kept on their own table. The functions.php on the tny.im folder has methods for adding and editing URLs that handle the tny.im features. It works like this: the code calls my custom methods for adding/editing short links; then, things are added/edited on the normal YOURLS DB table using the core methods, and then my code takes care of the other table which stores things like link hit limit and price (for internet toll links), as well as adding the link to the users’ account when appropriate.

Speaking of user accounts, the data that relates users to the keywords they have access to, is kept also separate from YOURLS tables. Same with the lists of links feature.

Finally I moved the columns of each database table to their correct destination in the new tables – five hours worth of SQL commands. Then I moved the new script files to the vhost of the live site, after editing the YOURLS configuration file and the nginx vhost config of course. It seemed to start working right away. All this took me two days.

Now the code for tny.im is much more clean, readable and most importantly, maintainable. I can finally add new features to tny.im without breaking half of the existing ones. And I can update YOURLS without breaking the whole thing, since none of the new features are implemented in the core files. @ozh’s improvements in pages like the links statistics one and yourls-loader will not automatically get merged with the tny.im code, but I can add them manually while keeping my changes.

I’m not yet fully sure the new tny.im is clear of bugs, but over 90% of it seems to work the same or better than before. It should look and work pretty much the same as it did before – which only proves, that I really made a wrong move when I started editing core files, since everything could have been done without touching them. I think I learned the lesson, hopefully not only for YOURLS, but for most scripts which have plugin interfaces.

I think I made the most full-featured URL shortener ever seen, and obviously I’m proud (certainly too proud) of it. It’s all built around YOURLS, and I bet you wouldn’t even tell it was powered by that script at a first glance. Stay tuned as more features and bug fixes are to come.

Utilities Beta 9 is out!

Six months after the previous public release of my Utilities add-in for the Casio Prizm calculators, the new Beta 9 is out! This version comes after an important refactoring of the source code, which by itself led to many fixed bugs and much higher stability. The g3a binary is also smaller (about 136 KiB) while featuring the same features plus some new ones, and that’s a big plus on a device with limited storage memory like the Prizm.

Changelog:

The most noticeable new “feature”, is probably the increased stability on tasks and calendar events, as well as in the file browser.

You no longer need to keep the storage memory fully optimized for calendar events and tasks to work without system errors.

Another new thing is a new timer/stopwatches system, which is now called “Chronometer”. There are 20 chronometers and each of them can go upwards or downwards. You can now select multiple chronometers (hint: Shift+F1 on the chronometers list shows useful selection options).

There’s now also a way to open files as text, with automatic line ending detection (supporting CR+LF, CR and LF) – on the file manager, press EXE when hovering on a file, then press F1 on the file information screen to open the text viewer. This still doesn’t automatically convert special characters from ASCII to Casio’s multibyte system but it’s something I’m planning to add.

File copying has been added, but this is unfortunately still too unstable, so this option will only appear when the “Show advanced tools” setting is enabled. Note that this is one of the last two functions of the add-in which are unstable (the other is the Add-In manager, which keeps being unstable probably because of a OS limitation – one can’t touch the add-in array under normal circumstances…).

Now there’s a way to save the current time to a OS Basic variable (those you can use on Run-Mat and almost everywhere in the official software): press the X/θ/T key on the main screen (where you see the time), and the hour fraction will be saved to the variable T with the maximum precision the RTC allows.

This add-in is finally headed for a stable 1.0 release – this Beta 9 release would be called a RC, if it weren’t for the Add-In manager and the file copying function, as well as a little feature that was committed to the git repo already in March, but got lost as I refactored the code. I shall add that little feature in time for the next release.

Update instructions:
(only applies if you used Utilities before Beta 9)

This bit is really, really important as there are some folder names that changed with this new release; also, not all files are compatible (calendar events and tasks are compatible, though). You must follow the following instructions for Utilities Beta 9 to work properly:

Delete the @UTILS folder in Main Memory (Memory app in Main Menu, press F1, scroll down to @UTILS, press F1, F6 and then F1);
– Now connect the calculator to your computer through USB, as a flash drive, and perform the following steps:
Rename the @CALNDAR folder in the storage memory of your calculator to @UTILS (you must do this on the computer because the OS won’t accept the @ as a valid character);
– Inside the newly renamed @UTILS storage memory folder, delete the file Hash.plp in case it exists (the new Hash.plp file is no longer compatible with the ones from previous versions).
– Copy the new utilties.g3a (download below) to the root folder of the calculator, overwriting the old one.
– Safely disconnect the calculator from the computer. You are ready to use the new version once your calculator finishes updating the Main Memory…

Download Latest Version of the Utilties add-in for the Casio Prizm

Alternative IM/VoIP solution?

If anyone knows of a instant messaging and VoIP solution that’s free, privacy friendly (end-to-end encryption, preferably) and, most importantly, easy to get the “average Joe” to switch to (i.e. my non-technical friends), please let me know. Bonus points if it has interoperability to work with other systems (at the eventual cost of losing some of the privacy) – that way, I wouldn’t need to get all my friends to switch.

(Of course, Google Talks/Hangouts, Skype and anything Facebook-related is out of the equation.)

Another thing I’m looking forward to is having my computers (Linux and Windows), old Windows Mobile phones and my Android phone and tablet synchronized (contacts, calendar, email) through the Internet without Google’s help, but I realize this is even harder find than the IM solution I specified above.

Slitaz servers down

Seeing that I’m getting a lot of hits on this blog from people searching about the SliTaz servers being down right now, probably because I have a blog post with the words “SliTaz” and “dead” on the title, which was published over a year ago, I’d like to inform that I know nothing about the server situation other than the fact that the website is unavailable for me too.

(talking about SliTaz, I kinda stopped following the project, even though I find it interesting… note to self: pay more attention to it from now on)

Updates on lots of things

I figured out I didn’t post anything here for quite a while – actually more than two months! There have been a series of events that have stopped me from updating this blog and, for a good part of these two months, have stopped me from doing anything besides checking my email and doing some school work.

The first cause for my online absence was that, a few days after I posted about this server having been reloaded, school classes have started. I happen to be taking probably the hardest school year I ever took – at least, that’s what older people said before it started, and which proved to be true now that two months of classes have gone by. Free time isn’t that abundant and I prefer to spend it doing actually something more useful than writing about random things on this blog. That almost explains why I haven’t been active on communities like FreeVPS. But there’s more…

Some time in mid-October I was left without a reliable, 24/7 Internet connection. This, after two or three months without an unlimited-traffic Internet connection, during which I had to hold the downloading of system updates for both Linux and Windows on all of my machines, plus the Android tablet; to keep to a minimum the access to online streaming media; and of course to hold any other big download/upload tasks. I had no Internet connection at home, which means checking the emails at school when possible. Checking anything other than email is horribly slow and unpractical on my old phone, and that’s why I repeat email is the fastest and most reliable way of communicating with me.

At the same time, I was being kept busy with school work. Free time was spent developing an Utilities software for my graphic calculator, a Casio fx-CG 20; this was probably the most useful thing I could do without an Internet connection, because at least gcc still hasn’t moved to the cloud.

Around 1st November, the technical problems that were stopping me from having Internet at home were solved (read: free wifi hotspot magically came back online!), and so the second cause for my lack of participation in the online life was solved. But there was still the first cause: school. So slowly, as I was having free time, I began to put me up to date on what happened while I was offline, and to make slow progress on my projects.

Apart from all this, I also have friends and family, and my real-life social life is way more active than it used to be one or two years ago (my brain must have installed some system updates, finally).

To conclude, on the IT side of things I’m now developing a JPEG image viewer for the Casio Prizm, using the picojpeg library (incredibly useful). The thing already works but is damn slow, plus selecting files is still a pain; this Prizm add-in isn’t yet published anywhere, or even announced on any place except this blog post (and don’t consider this an official announcement).

I finally had time to fix some bugs at tny.im and add some features: the optional short URL toolbar is now powered by Meny, which was developed by a guy which also has lots of other awesome work, namely a promising online presentations service called rvl.io.

I’m also trying to keep participating on online communities such as FreeVPS and Cemetech, and to keep my Twitter feed rolling, but the first cause for my online absence is still valid, and will only stop being on the beginning of July next year. This doesn’t exactly mean I won’t be online, but that you shouldn’t expect me to be as active as I once used to be. Again, if you need something, email me (admin at tny dot im, or my username on Gmail).

Until next post!

“Hidden” 256MB storage on the Flytouch 3 P041

I was playing around with my cheap Flytouch Android tablet, using dd to create images of the different partitions of the internal storage (which is, in fact, just a microSD card).

Turns out I discovered that there is a 256MB FAT partition living on /dev/block/mmcblk2p6. By this time, geek users already know what to do: with root privileges, mount the partition in some directory..

So, open a terminal on your rooted Flytouch 3 (P041 and DK1031 models should have this partition). Type:

~# mkdir /mnt/sdcard/256MBfat
~# mount -t vfat /dev/block/mmcblk2p6 /mnt/sdcard/256MBfat

This should result in a new folder in your sdcard directory. This folder is a filesystem node, like the sdcard1, udisk1 and udisk2 folders.
This FAT partition is empty and should have about 256MB space. If it isn’t, or if the mount command returned an error, then that’s interesting 🙂

Why is this partition in these tablets? Well, I have a theory. These tablets support having a recovery partition, even though most firmware updates provide no recovery image files. The fact that this is a 256MB partition may indicate that it is meant to hold contents similar to the system partition, and in this case, it should be formatted as EXT3 and not FAT.
As most firmware versions for this tablet available on the internet don’t include a recovery image, this partition just gets formatted as FAT by the updater kernel at update time.
But what if the updater never touches this partition? This would be pretty good news. You could use this more or less hidden partition to store the owner information, so if your tablet ever gets stolen you’d always have a way to recover it.
If that was the case, you could also use this partition to store essential APK files and configuration so it would be easier to recover from a firmware update or factory reset.

As a last and kind of unrelated statement, I’d like to point out that the bootloader of InfoTM tablets is much more complex than it may appear at first.
Through a serial line that I believe to be the A-A USB connection used by IUW to burn updates, the bootloader can provide a serial console, that can be used to change the boot parameters for Android and maybe even boot other operating systems from the external SD card.
This thing of the bootloader is something I’m figuring out slowly by analysing the uBoot update file I have, u-boot-nand.bin.

If you have one of these InfoTMIC tablets, feel free to comment below with any important additional information.

NOTE: this post was written months ago but was sitting on a text file on my desktop for months, waiting to be posted. So this isn’t a recent discovery, but still an interesting one.

Presenting tny.im

Yesterday I wrote a post saying l.f.nu was down… and it still is.
So I bought a domain and moved my URL shortener to yet another domain:

http://tny.im

This time, I’m sure it will be up for at least one year – if it goes down within this period, at least it won’t be because of the domain, as that’s paid for an year already. I didn’t pay it, some friends at Cloudstg did – I’ll pay them back gradually, by advertising their services and such. Again, thanks for investing $11 on my service: if it weren’t you, I’d have to spend my savings on buying this short domain, which would leave me with no money to renew this .com domain next October.

The tny.im domain is as long as l.f.nu, but with less dots, nicer, and since it’s a top level domain and not a subdomain, I have much more control over it. This is a important point, as I plan on adding IPv6 support to tny.im, and a FreeDNS subdomain wouldn’t let me have multiple records on a subdomain. With a real TLD, I can have both an A and an AAAA record for the same domain.

Like it was with the transition from 4.l.to to l.f.nu, no data has been lost, and 4.l.to and l.f.nu links work as long as you change the domain to tny.im. Statistics, link editing, etc. all work.

I hope you enjoy tny.im, and remember, this time it’s for real: the shortener will be around for more than a year, assuming I can get enough profit from it to keep paying for the domain. Having me profiting with tny.im only depends on you – by using my shortening service, you’ll help me earn some cents from ads (but, please, don’t click-bomb them!), which I’ll use to renew the domain and eventually pay for server(s), in order to offer you an even better service.

Again, I hope the ads are not annoying… if they are, make sure to drop me a line so I can fix them.

Casio Prizm software

Own a Casio Prizm? Please take five minutes (probably less) to fill my survey on product IDs. It’s as easy as looking at the back of your calculator and typing 15 characters. No personal information is collected. Thank you!

I developed some add-ins for the Casio Prizm series of graphic calculators (fx-CG10 and fx-CG20; some add-ins are compatible with the fx-CG50 and Graph 90+E). The software is available for download directly from a server of mine, sometimes from Cemetech and sometimes from Casiopeia.

Utilities

clockeditor

The Utilities add-in is exactly what its name says: an add-in full of utilities not originally found on the Casio Prizm calculators. This includes:

  • Clock, with time and date and the tools for adjusting them;
  • 20 chronometers (can go up or down) with accuracy as good as the 128 Hz tick allows (it’s better than 1/100 of a second);
  • Tool to fine-adjust the backlight level (from 0 to 244, versus the 1 to 5 provided by the OS);
  • Tools to adjust the poweroff and backlight timeout to values not possible through the OS default tools;
  • Power information (read battery voltage level, power source, battery setting and more);
  • Many lantern types (make an expensive and inefficient lantern out of your Prizm);
  • Calendar with agenda (add, edit, view and delete events on days, search events, week and month views, and much more);
  • Task list (add, edit, view and delete tasks, which are basically calendar events without dates);
  • File manager (rename, move and copy files and folders, with long-file-name support and non-capital letters support). Also allows for opening text files, compressing and decompressing files in a special format, and searching for files;
  • Text editor (create and edit text files);
  • Balance manager (manage your pocket money);
  • Password generator;
  • TOTP authenticator (like Google Authenticator and compatible with it, but on a calculator);
  • Graphical memory usage viewing tool (check how much Main and Storage Memory is used with nice progress bars);
  • Add-in manager (choose which add-ins are available for launching in the Main Menu without having to delete them);
  • Function key color selection (select the color of the function key labels throughout the OS; I recommend blue if you’re tired of black);
  • System information viewer – view the OS and bootloader version and timestamps, calculator model information, Renesas CPU version registers and the unique device ID.
  • Calculator lock with numeric code (like CGlock but more advanced);
  • A way to save the current time as a decimal number to the OS alpha variable T (you can use this on Run-Mat and almost everywhere by pressing ALPHA and then the division key).
  • Lots of settings to make the add-in behave the way you like.

All this in a binary file under 185 KiB!

Version 2.1 of this add-in was released on June 21, 2016, with this most likely being the last release; at this point Utilities is stable and full-featured, as you see in the above list. New releases will only happen to fix bugs, without increasing the feature list. I have tried very hard to make sure this add-in provides something useful for everyone who owns a Prizm.

Download g3a

Download ZIP of edition with support for running PicoC scripts – More information about PicoC

Thread at Cemetech

GitHub repo

This add-in runs on the newer Prizm models fx-CG50 and Graph 90+E, and most things appear to work, but it has never been officially tested on these models. As always, use at your own risk.

Image viewer

View JPEG and PNG images on your Prizm. Progressive or lossless JPEG and interlaced PNG not supported. Has strip functionality for linking (not embedding) images on eActivites. Stable version 1.1 is out! More information on the Thread at Cemetech, or download the g3a – just copy to your calculator (out of any folder!) to install.

This add-in supports the newer Prizm models fx-CG50 and Graph 90+E.

Eigenmath port

Screenshot of my Eigenmath port

Screenshot of my Eigenmath port

I ported the Eigenmath symbolic math engine, by George Weigt, to the fx-CG 10/20 calculators. Version 1.0, stable, is available for download, with all the results matching the ones given by the PC version. Updating is heavily recommended, because there are lots of new features in comparison to the first release, not to mention more ability, stability, and many bug fixes.

This port includes a function catalog, ability to create and execute scripts, draw graphs, and much more. All 100+ Eigenmath commands work. Output is pretty-printed whenever possible, and 200 lines of output are stored for scrolling at any time. Command execution can be stopped. The state of the add-in is optionally automatically saved when exiting. This add-in can also act as a eActivity strip.

Download or find more, updated information on the thread at Cemetech. GitHub repo.

For the time being, this add-in does not support the newer Prizm models fx-CG 50 and Graph 90+E.

Discontinued software:

These add-ins do not support the newer Prizm models fx-CG 50 and Graph 90+E.

CGlock

An add-in that lets you protect your calculator with a 4-digit code. It can be set up to automatically turn off the calculator when it is locked, to not show the Casio logo (and consequently the owner information) at turn off, and to open Run-Matrix after the correct unlock code is entered. If the Prizm is rebooted or the batteries are taken off, the calculator goes unlocked – so this program only provides mild security for your calculator (you can always epoxy the reboot hole and the batteries compartment, but do so at your own risk).

It saves the code and the settings in the main memory. I have some ideas for new features, such as hashing the code for extra safety and implementing CGlock’s own owner information display, since the Casio one is easily crackable, which I’ll implement when I have time and feel in mood.

Download from Cemetech

OverClui

OverClui is a tool for overclocking the Prizm’s CPU based on the work of Ashbad and brijohn at Cemetech. The difference between this and their own tool, Pover, is that this one has a nicer GUI, and is more noob-friendly. It doesn’t let users overclock to 101.5 MHz but only to 94.3 MHz to avoid problems, even though recently I found out some calculators don’t handle the 94.3 MhZ speed and thus can only to 87 MHz safely.

Use at your own risk, no warranty provided. If the calculator shows a “SYSTEM ERROR” message when overclocking to 94.3 MHz, please press EXIT to reboot, don’t overclock to that speed again (which also means you won’t be able to run the latest version of the game Gravity Duck on your calculator) and let me know in the comments.

Download from Cemetech

Status update

I have been very busy with my offline life: school, family and friends haven’t been leaving much time left for me to blog here. When I have some free time, I try to keep up-to-date with the online communities I take part in and also work on my l.f.nu URL shortener. By the way, have I told you that l.f.nu now supports editing short links?

When you shorten a new link, you receive a random code specific to it. Keep that code saved as if it were a password, as it is the only way to edit a shorten link through its Click Statistics page (add a + symbol to the end of the shorten link, then open the tab “Manage”).

This feature about link passwords (which I call “passcodes”) is something I developed just for l.f.nu, it is not available in the standard YOURLS installation. I have no plans to make it open source right now, as I haven’t implemented the thing as a plugin, and the code is a bit unorganized.

So no, I haven’t disappeared from the online world yet. I’m just a bit more silent these days…