Internet forums… websites and communities where I spent thousands of hours. To me, they already feel like relics of the past. I’m 21 and began using the internet about 10 years ago, maybe more. I’m sure the majority of kids who are now starting to browse the web (probably at earlier ages than I was) will not participate in a classic forum, perhaps not even land there through some random search. You know, the kind of discussion websites where there are threads, posts, moderators and rules, things are typically ordered in chronological order, and reactions to posts (“votes”, “likes”, and similar) are either unavailable or something added to the system as an add-on rather than a core mechanic.
In recent years, even I – someone who used to be staff on a medium-sized forum, FreeVPS – have withdrawn from most forums where I used to actively participate. Many of my friends who know what a forum is, either never got to participating in one, or are now participating much less than they used to do two, three, four years ago.
In recent months, it has come to my attention that some of the forum-based communities I frequented are facing serious drops in activity levels, sometimes to the point where days go by without a post, where previously there used to be at least one post per hour. While this reduction in activity is in part because the main topics of those forums are becoming more and more niche, I believe a big chunk of the problem has to do with the fact that they are, well, forums. The lack of activity has even led to meta-discussions on new directions and ways to bring life back to these communities – see the threads at Cemetech and CodeWalrus.
This is the first post of a series of blog posts where I’ll look into the state of forums in today’s web, identifying what alternative discussion formats and platforms exist, why users seem to prefer (or hate) them, and what is the role of forums nowadays and what they can still do better than other mediums. With this being a blog and not a scientific study, I will obviously write from my own experience and the experience of my peers, producing subjective, unverifiable but hopefully enjoyable content. I’ll also provide my opinion on why forums are still necessary, and give suggestions on how forum-based communities become more appealing to today’s web users.
With this, I hope to be able to bring some life and schedule to this blog – blogs, which are another medium that is nowhere as popular as it once was… maybe I’ll look a bit into that too. This post will act as the index of the series, so links to new posts will appear here once they are published. Stay tuned.
If I haven’t posted much here, it’s in part because the masters degree I’m pursuing is quite demanding, and in part because my “creative writing” time is often spent replying to interviews for Portuguese national media (and dealing with the exposure, which sadly is not expo$ure) about my UnderLX project, over the last few months:
…and with more to probably come 😉
A few years ago, I was very active in the Casio Prizm development community, having developed three notable add-ins, contributed to the Prizm wiki, libfxcg (my fork), and even done a bit of reverse-engineering (the calculator OS is closed-source and there is no official SDK), that resulted in the discovery of a couple of syscalls and more detailed documentation on some other ones. Because of this, once in a while I still get messages about my add-ins, which I’m happy to support when possible. Most annoyingly, I also get messages about Prizm development, usually about how to start making add-ins.
Why are these messages annoying? Because I don’t really know how to answer. When I started developing add-ins for the Prizm, I had little to no knowledge of the C programming language, and yet, despite the fact that add-ins can’t make use of all the stuff “normal” C programs can (the libc provided by libfxcg is incomplete; the filesystem uses a different API, there’s no threading, the stack is giant compared to the heap, etc.), I managed to learn it. It certainly helped that I had some previous experience with programming in other languages, even if it was just sloppy code, but I don’t have much of an idea of what to say to someone who intends to learn programming using the Prizm.
I usually end up saying that learning programming using the Prizm it’s a bad idea, probably coming across as extremely discouraging. However, I do hope it’s for the best, and that these people will still learn programming – just not by developing add-ins! Had my first contact with programming been through Prizm add-in development, most certainly I would have chosen other career path than computer engineering. I mean this seriously. I’m glad my first contact with programming was through sloppy Visual Basic code. Anyway, I already wrote a post on my programming experience – it needs updating, but it should do.
Learning how to program, even in a “easy” language and common platform, can be overwhelming; for a programmer that is used to higher-level programming, learning the Prizm, a poorly documented platform with a small developer community, can be overwhelming; combine learning how to program with learning a poorly documented embedded system, and it will most likely be very overwhelming. Of course, nothing will stop someone extremely motivated – hopefully, not even my less encouraging replies, or this blog post.
What follows is the partial reproduction of an email I recently sent, in reply to yet another of these inquiries on how to start developing for the Prizm. I have edited it to make it less specific to the situation of the person I was replying to. I’ll also use the terms “Prizm” and “fx-CG 50” interchangeably, as add-ins built for the former run, with a few exceptions due to sloppy coding (one of mine’s one of these exceptions…), on all Casio Prizm models: fx-CG 10, fx-CG 20 and fx-CG 50.
Do you have any previous programming experience? If not, I honestly do not recommend starting with the Prizm or any other Casio graphical calculator. If yes, then be aware that this is not an “easy” platform to develop for. Either way, here are a few reasons why:
- Prizm add-ins are written in C or very limited C++ (that might as well be considered C). By today’s standards, these are very low-level languages that require manual memory management and a very good awareness of the machine. They also provide very little protection from programmer mistakes. While some people had C as their first programming language, it is by no means a beginner-friendly language.
- Even if you already know C well, or if you learn it from any common book, tutorial or course, you’ll be disappointed to find out that much of the standard library is not present, or is insufficiently implemented, in the Prizm calculators.
- Add-in development for the Prizm was made possible through reverse-engineering and educated guesses based on what was known about previous models like the fx-9860G. While we now understand the essential things about the OS on these calculators, many things are yet to be known.
- Documentation is lacking and the community is not very large. This essentially means that you won’t be able to just google your way through many problems.
- Reverse-engineering/documentation and development efforts for the Prizm have basically stalled. You’ll also find many materials that mention the fx-CG 10/20, but since the fx-CG 50 is basically just a faster version of these with a mostly compatible OS (although some things like memory addresses have changed), almost everything you’ll ever need will still apply.
Now, I’m sorry if I came across as dismissive or as discouraging, I’m just trying to make sure you know what’s in front of you.
For Casio Prizm development specifically, this is where I can point you:
Prizm forums at Cemetech
Use these to ask any questions you might have and try to find solutions to any problems you encounter. There are also some guides there, mainly on how to set up the development environment (compiler and such), but I’m afraid they might be a bit out of date. However, as I said, development efforts have mostly stalled, so consider anything from 2014 or early 2015 as up-to-date. Specifically, do not follow the “[HOWTO] Prizm C Development” in there, as it is out of date.
This wiki contains much information on the calculator, the reverse-engineered OS functions (“syscalls”) that can be used from add-ins, etc. It also contains more up-to-date instructions on how to set up a development environment.
Personally, I have mostly moved on from Prizm development about three years ago, as I began pursuing a degree in Information Systems and Computer Engineering. Every year or so, I make a short comeback to fix urgent issues with my add-ins and eventually make them compatible with new OS versions and calculator models, as is the case of the fx-CG 50, as long as that does not require too much time/effort. As time passes and I work with other technologies, the more I realize more how “hard” of a platform the Prizm is, and the less motivated I am to build stuff for it again; the fact that I no longer use my fx-CG 20 nearly as much since high school, also doesn’t help.
I’m afraid I can’t help you much more, as I’ve forgotten much of what I knew about the Prizm, both the “theoretical” and “practical” knowledge, and I no longer have practical access to a development environment for it. I tried to put as much of my knowledge as I could into the Prizm wiki before I left, and I believe that the people that now frequent the Cemetech forums will be able to help you much better than I can.
I think that one day I might find some interesting in working on the Prizm again, but perhaps more from a reverse-engineering angle. As for the fun in developing for a constrained, embedded system, there are much more appealing constrained systems out there, like the ESP8266.
I neglect this blog very much. I write a non-negligible amount of stuff on the internet, but it’s usually on forums, Reddit, Hacker News and similar, and this blog ends up forgotten. I just wrote a 2000 words post on at CodeWalrus, about the current state of Clouttery, but since it also contains a personal-life-log part, I thought it would be the perfect thing to cross-post here. It’s not the first time I paste forum posts here, and it’s something I should probably do more often, as it helps keep this blog alive while at the same time preserving posts that, while usually made as replies to a forum topic, are general enough to stand on their own.
Hello everyone again. I figured it was time for an update, even though this is not exactly a “happy” update, at least as far as Clouttery is concerned. This is a long post, bring it to bed so you can fall asleep to it if you wish, but trust me, it’s worth reading. I hope you can learn a thing or two about managing your side projects, from reading about my mistakes.
Last school year was the last year of my undergrad course (and I’m starting a second cycle course in a couple weeks) and this required some more effort, so I had less time for side projects. As often happens when one works on something for an extended period of time, I too gradually lost interest in this project.
To make things more… interesting, in mid-March I launched a small website that was meant to be kind of a practical joke about the unreliability of the Lisbon subway (for those who haven’t yet figured it out, I’m Portuguese). You’ll be able to understand what it is about by checking out its repo on GitHub.
I started that project mostly to have something different to work on that was not Clouttery, and the original plan was for it to be something I’d build in a few weeks, publish and then forget, for it to be yet-another-small-thing in my portfolio. But to my surprise, after minimal “marketing” on the SkyscraperCity Portuguese community, that has a section dedicated to railways and subways, my website received a lot more attention than I was expecting, especially for something so simple and tongue-in-cheek.
I then understood there was a real interest in a service that would let people work-around the problems in the Lisbon subway, while at the same time denouncing the problems with the service (for example, by collecting independent statistics). Long story short, a small community assembled around this project, which ended up evolving into an Android app called UnderLX that’s even published on Google Play. And there’s still a lot of work to do for it to become the product I envisioned.
This obviously took most of my summer.
Yes, it’s true that, unlike Clouttery (for which I had even written a complete billing system from scratch!), I’ll never be able to monetize UnderLX effectively. However, it is way more satisfying to work on, at least until I get saturated of it too. With Clouttery, it often took a while to realize what it was about, and let’s be honest: the final reaction of many people was just “meh”. However, with UnderLX, people tend to pay a bit more attention, and those who understand the whole potential of the project usually become much more involved in it. “Unfortunately” for me, the technical side of it is more complex than Clouttery.
It also has an “advantage” to my eyes: both the client (Android app) and the server are open-source from the first day. I regret not going this route with Clouttery; now I have lots of closed-source code which I can’t easily show to anyone because, well, it’s in private repos. I’ll go back to this point, later.
Finally, to add to the school work, the gradual loss of interest, this happy accident that was UnderLX, there’s a fourth factor in all this. Because of multiple reasons including the astronomical rise of the price of Bitcoin, it made economical sense to fulfill a long-time desire of mine: to build a desktop, so I could have a powerful machine, more powerful than my six-years-old laptop. Picking parts and putting it together was very enjoyable, and now I have a proper workstation like I had been dreaming of for the past couple years. If you are interested I might even post a thread about it here. I wasn’t much into PC gaming before (in part, because the hardware didn’t really allow for it), but… you see… to sum things up, many hours were spent chilling to some great triple-A titles (thanks Steam summer sales!…).
That’s all really nice, but I thought this topic was about Clouttery?
Work on Clouttery gradually slowed down through the last months of 2016, subject to how busy I was with school, and pretty much completely halted in March this year, as I got more and more tired of working on it, so I decided to do that “small” subway thing. It also didn’t help that I was going through a complex phase with Clouttery, more specifically regarding the Windows and Linux clients.
- I couldn’t get the Linux client to work right, despite trying to develop it from scratch multiple times using different languages and technologies. No matter what I tried, there were always major roadblocks to getting it to the point I wanted. I did not want to write it in C or C++; I hate Python but decided to try using it anyway – didn’t end well; UI framework bindings for Golang are apparently all terrible, or don’t have a suitable license. Mono would have been a viable choice, but the gtksharp bindings were tricky to get to work, they apparently had to be recompiled for each GTK version (meaning I couldn’t simply distribute a single binary), and the bindings for GTK3 aren’t/weren’t exactly ready for prime-time. Ugh, I don’t even know what all the problems were anymore. I do remember that I just wanted to write code, but problems with libraries and bindings and whatever were always getting in the way.
- The UI of the Windows client suffers from major lag and other problems, so I decided to switch from WinForms, which is no longer supported, to the supported and way more modern and flexible alternative: WPF. But this was taking a lot of time, certain things were much harder to get to work than I expected, it didn’t help that I only had time to work on it like one hour at a time (school work, etc.), and I lost more and more interest.
For you to get an idea of how inactive this project has been, these are the dates of the latest commits to Clouttery repos:
- Server: 2017-07-10 (and this was only to fix a bug with notification filtering; previous “real” work on it had been on the 23rd of March)
- Windows client: 2017-03-26
- Android client: 2016-12-29
- Chrome client: 2016-09-08
- Linux client: 2017-03-09
Earlier, I mentioned I regret not open-sourcing Clouttery from the beginning. I decided to work on it privately, because it was supposed to become a commercial service, and I feared that if I made it possible for people to host their own Clouttery servers and recompile the clients to talk to it, then nobody would pay for the service. This is obviously a stupid way of thinking, especially when the project in question is a personal project of a student that doesn’t have much time to work on it, and likely would never be able to get it to a commercially-viable state. If the service was worth it, I guess most people would happily pay for it, just to not have the hassle to figure out how to make the server and clients work for themselves; this is especially true since the target audience wasn’t exactly software developers nor sysadmins, i.e. it was people who wouldn’t have a clue how to do that nor would bother even if they were given clear and easy instructions.
Right now I have 30K lines of code, possibly more, that’s closed-source, but for no good reason. To aggravate things, Clouttery shows more of my abilities in software development and engineering than any of my open source projects, because it contains code in more languages, for more platforms, than any other of my projects; it includes web design, API design, use of cryptography, etc. It is not the most beautiful code (for example, the UnderLX Android app has much cleaner and organized code than the Clouttery client, and even then it’s not exactly stellar), but it works, and definitely shows what I’m capable of.
This whole situation is even more ridiculous, because right now there’s very little to no code in Clouttery that’s “novel” to the point of requiring intellectual property protection. At this point, Clouttery is extremely dumb, as I never got to work on the “intelligence” that would involve machine learning, pattern matching and the like. And in the end, if I wanted to make my super-awesome-and-courageous battery level prediction and damage identifying algorithms secret, I could always have added them as a closed source module while keeping the “infrastructure” open source.
Finally, I always told people that if I were to stop working on Clouttery, I would release its source code. I don’t know what you think, but if I half-close my eyes and look from far away… yeah… like that… yep, I definitely stopped working on it.
Then why don’t you open source Clouttery?
I definitely want to open source Clouttery, so I can show its code to more people, and so that others may eventually try to pick up on the project. I intend to keep running the official Clouttery server – if for no one else, for me, as my family finds Clouttery useful. I think I would be a little sad if someone took the project and simply changed its name and started running their own “official” service, available to the masses and possibly profiting from it, so I’ll see what kind of licensing restrictions I can add to prevent that. Without the help of a lawyer, it’s a bit hard to add clauses to existing software licenses or write new ones from scratch; even with legal help, it’s easy to get to a controversial result – for example, Facebook has that famous problem with the “patents” file on their open source projects, like React.
…so why didn’t you do it already?
I also need to write a bit of documentation explaining essential things about each repo, and ideally also explaining how to run the server, what needs to be in the database, etc. Or I could not care about any of that, and just have people figure it out by themselves – at that point, I make it hard for people to contribute, but at least people can already look at the code, which is much better than the current situation.
As you can imagine, all this takes time and effort, and I’ve been busy with all those things I mentioned in the first section. But with classes starting very soon, I’d like to get this going ASAP, or it’s not going to be done for some more months.
Did I “give up” on Clouttery too soon?
Is it a good idea to open source it, even if in the distant future I decide to turn it into a proper commercial service?
Do you agree it would have been better if it were open source from the beginning?
What license do you think would be most adequate? – don’t forget the server and each client can have different licenses.
Would you be interested in submitting pull requests for this project, perhaps even taking over one of the sub-projects or the whole thing?
Share your thoughts.
I spent the past week, the last one of my winter break, redesigning how the Clouttery server stores data.
The Clouttery server, which is written in Go, was using a simple key-value store (Bolt). I slowly came to the realization that some of the features on the roadmap would be kind of hard to implement using Bolt; that the nested buckets structure used with Bolt was too limiting, by forcing a hierarchy on the data, when sometimes it could be useful to interpret it in other ways. For example, sometimes it could be useful to look at all the battery log entries from all users; with the database structure I had, that required looking into each user’s bucket separately, and within those, into each device separately.
The databases course I took last semester forced me to get my hands very dirty with SQL, and after seeing the benefits, I decided to move to a relational database.
Another reason for moving was that Bolt can’t scale (no replication, it’s meant for use by a single app, like SQLite), and while the server software is not yet ready to be clustered, moving away from Bolt (and, in general, uncoupling the server from the database) is a giant step towards the goal of being able to scale the server to multiple nodes. I had known for long that I had to use something other than Bolt if I wanted to make the server distributed, I just wasn’t sure whether to move to a relational database, another barebones key-value store, or some amalgamation of solutions involving specialized time series databases or what-have-you.
The database can now be accessed transparently by multiple applications, which means that, for example, in case I want to do some complex analysis on the battery histories, I no longer have to stuff that code into the server. I can even use a language other than Go, like Python, which I really don’t like, but has many libraries for data analysis.
I tried to use CockroachDB (and I can’t stress the terribleness of that name enough). At some point, the server was mostly ready to work with it, and it was time to import the data from the Bolt database. My code migrated all data in a single transaction, that was rolled back in the case of errors – that way, as I stumbled upon problems and general incompleteness in my migration code, I did not have to be constantly dropping and recreating the database, as with every failure the database would be always supposedly in a pristine state, with all the empty tables waiting for data.
Let’s just say things were not as smooth as I was hoping. On my laptop with an aging but still plenty fast i7, 8 GB of RAM and a SSD, data would get into CockroachDB relatively fast… but no matter if the transaction was committed or rolled back, once I tried to perform any query – basically any query, even if just counting the amount of users (about 40), would make CockroachDB’s RAM usage skyrocket, to the point where the whole system just hanged for seconds at a time, due to how much swapping was going on.
So I decided to scrap CockroachDB and go with plain old PostgreSQL. Given that the SQL supported by the former is relatively similar to what PostgreSQL supports, changing the queries to work with Postgres was not too hard. The most annoying part is the lack of support of PostgreSQL for the UPSERT command, which in CockroachDB and other databases, behaves like a INSERT when there’s no uniqueness conflict, and like a UPDATE when there’s a conflict (in which case it will update all the other columns). I had about ten UPSERTs that had to be rewritten as INSERT … ON CONFLICT (…) DO UPDATE SET – followed by all the columns to update. Ugh.
Importing data into Postgres was noticeably faster than into CockroachDB, and most importantly everything kept working fine after about a million entries were in the battery history table. And yes, everything was still inserted in a single transaction.
I took the opportunity to perform some long-needed changes to the data types used by the server. Making sure Clouttery clients kept receiving data with the formats and semantics they were expecting was a bit of a challenge, but very easy in the grand scheme of things.
As a very nice bonus, the server now does transactions properly. Previously, for a single API or website request, multiple Bolt transactions could be made. If something went wrong with one of the latter transactions, that one would be rolled back and no more transactions would be performed, but the changes done by previous ones would stay – like most databases, Bolt doesn’t let you rollback a committed transaction. Obviously, this could result in an inconsistent state.
Now, and after changing most functions in the server code to accept what can be described as a “transaction node”, each API request, web console request, or admin command works in a single transaction. Either there’s no error and everything goes through, or everything is rolled back. No more inconsistent data. sqalx was the library used to implement this.
The changes were pushed to production about two hours ago – after extensive testing on the staging environment, which unfortunately didn’t catch all the bugs. To identify problems, there’s nothing like dozens of devices running different clients and submitting different data to your server…
A few hotfixes later, everything appears to be working fine, but I’ll be keeping a close eye on the logs where, hopefully, all errors are logged. I say “hopefully”, because during testing I found out that the error return values (in Go, errors are values) from some of my own functions were not being logged, and some were completely ignored…
It would be great if over the next few days users could pay a bit more attention to the behavior of Clouttery, namely making sure that battery histories are updating as they should, and that notifications are generated when they should, according to their settings.
I’m probably a bit too much proud of this – at least, until I find a horrendous bug. This is how things should have been from the start, but at the same time, when I started this project, I did not know enough about relational database design to even do a mediocre job. So I went the easy, “no SQL” route and just used Bolt, which allowed me to get to something that worked, relatively quickly. And now I’m glad I could turn it into something better after about 60 hours of work…
I barely have time to work on Clouttery, and it becomes less and less of a commercially viable project as time goes by. It’s one of those projects that seems to never leave Beta status, and not for good reasons. But oh boy, the things I learn…
I was casually going through my GitHub repos and came across PicoRed, a server redundancy manager I developed, with the immediate goal of managing the DNS records for the tny.im domain. The tny.im shortener used to be hosted by multiple servers in a Round-robin DNS configuration. The idea was that as servers went online and offline (or underwent maintenance, etc.), the DNS records would be automatically updated to reflect which servers are currently serving a service, in this case tny.im.
PicoRed is the successor to mersit, which served the same purpose but was written in very unidiomatic Python and was much clunkier than PicoRed (which is written in very unidiomatic Go, but used fewer resources and was somehow more stable). PicoRed and mersit were completely peer-to-peer, and this is because I couldn’t afford to have a “master” server that was stable enough and which I could be sure would have three nines of uptime.
The idea behind those tools is everything but novel; container orchestration, for example, requires similar tools to be deployed. For some reason, perhaps ignorance, back then I decided to write my own. (For an example of mersit/PicoRed done right, see Serf). I don’t regret it, of course: I learned a lot about distributed systems, and while my terrible consensus “algorithms” (a complete joke) worked, they taught me why things like Paxos and Raft had to be invented. The main takeaway was, “it’s complicated”. So for PicoRed I decided to use a library by Hashicorp that handled the hard parts for me (and that’s how my unidiomatic Go program was “somehow more stable”).
Three paragraphs into this post, and I’m still writing the introduction… these three paragraphs about distributed systems are just warming up for what’s coming, which is me saying that none of those homemade tools are in use anymore, and it’s not even because I switched to something better: tny.im, and some other services of the TNY network, are now served by a single server.
How did we get here? Back in 2014, I was a huge proponent of distributing every single service across many cheap servers, instead of buying a proper, rock-solid, big and expensive server from a reliable company. In theory, horizontally scaling would let one handle big amounts of traffic and improve availability at the same price, or even less – sounds great, right? These strong opinions were backed by the issues I was having with my BlueVM server. But now, we’re back to zero redundancy… what changed?
Well, my opinion is still the same: I’ll take horizontal scaling over vertical scaling any day, and the more redundancy that’s fit to pay, the better. The problem is when horizontally scaling begins to hurt performance and reliability instead of helping it, and that’s exactly what was happening in our case.
tny.im, dotAccount, PrizmID and my WordPress websites (this blog and the TNY network website) are powered by an extremely uninteresting LEMP stack. A LEMP stack is one composed by Linux, Nginx, MariaDB and PHP, or in other words, a LAMP stack but with Nginx instead of Apache. Until a few weeks ago, the “M” in this stack had the peculiarity of actually being MariaDB configured in master-master replication mode. What this means is that MariaDB was running on multiple servers, managing the same databases, and whenever a change was made, it was propagated to all of the other servers in the cluster (up to a few weeks ago, two servers; at some point in the distant past, up to five servers were used).
That’s how tny.im was served by multiple servers: simply by running the same PHP code in all servers, and having that code talk to the same database, replicated across all the servers. Of course, MariaDB master-master replication has its disadvantages. For one, performance is worse, because all database writes involve communication between the different MariaDB servers. This began to show on more database-intensive applications like dotAccount.
Perhaps more surprisingly, reliability is also worse. Perhaps I didn’t have MariaDB replication properly configured (after many attempts and hours spent, trust me), but it would sometimes break in wonderful states such as “WSREP has not yet prepared node for application use” whenever there was some network hiccup. This could happen as often as once a day, or once every two months (yes, networks are unpredictable like that). Whenever it broke, it would need to be manually restarted, and it would sometimes take multiple attempts until all the servers had their MariaDB running. In other words, exactly the opposite you want for a reliable system that requires minimum amounts of human supervision.
Perhaps PicoRed could have expanded into taking care of restarting the cluster, but since I couldn’t even get to a sequence of commands that, when executed on all servers at the right times, would reliably restart the MariaDB cluster, I kind of gave up. Lack of time and more interesting projects to develop, like Clouttery, meant that some stuff would inevitably get left behind, and my horrible mess of code called PicoRed ended up forgotten and eternally unfinished. Moving to proper solutions like Serf also required time that I didn’t have.
A few months ago I was notified that the provider of one of my VPS was closing, and all servers would be shut down by December 4th. I bought a new server, moved the stuff that wasn’t hosted anywhere else to it, but I really didn’t feel like reconfiguring the MariaDB cluster and PicoRed for the new server. PicoRed, in fact, stopped working in one of my servers (the one that wasn’t getting shut down) with some binary incompatibility error, a year or so ago. So I kind of gave up… reconfigured MariaDB so it stopped being a cluster, got rid of PicoRed, and said goodbye to one of the servers.
The new server is PHP and MariaDB/MySQL-free, and this probably won’t change. I would really like to move on from PHP and MariaDB to better languages and DBMSs. My main conclusion from the whole replication story is that MariaDB is not really prepared to scale horizontally, at least not without a lot of effort and “baby-sitting”.
I certainly have not given up on horizontal scaling, but I think that from on now, it’s best that I manage scaling at the application level instead of the database level, or alternatively, use a DBMS that was designed with horizontal scaling in mind, from the start. For the second option, it’s unfortunate that both CockroachDB and TiDB are still in a very premature state for production use.
I would rather not give up on relational databases; while it’s true that other types of database also cover some of the use cases of relational ones, I’m yet to know of any problem other than document storing that can’t be effectively solved with relational databases. (And for document storing, may I interest you in a relational database coupled with this strange thing called a filesystem?) Commercial solutions are obviously out of reach for me: it’s not like Segvault is a money-making machine; tny.im isn’t even profitable, despite all the ads!
I have grown to hate the mess of PHP and SQL that is tny.im so much, that shutting down the service (or at least getting it into a “read-only” mode) was once a topic for discussion at one of the TNY network meetings – all three of them. By the way, Segvault/TNY network is “hiring”, i.e. looking for new members with exciting project ideas, and if you had the patience to read this post this far, you may be a good candidate – contact me somehow.
Ads at tny.im earn me pocket change, that is used to offset the cost of the servers and domain names, and this shall be enough motivation to keep maintaining tny.im and supporting its users for some more years, updating MariaDB one version at a time.
Today, 500 days have passed since the initial release of Windows 10. I quickly and unscientifically reviewed it right after it was released, in two blog posts: one mostly complaining, and another mostly praising it (as if I was seeking some sort of redemption). The former one was a huge success, if we take into account the readership numbers for this blog. That post accumulated over 30 thousand views in the few hours after its publication – and a month later, we were back to our usual readership stats of approximately zero views per hour.
But don’t get fooled by these yuge numbers; I’ve probably spent more hours of my life using Linux than Windows, which probably means my opinion on the latter actually isn’t worth shit – but don’t worry, as I’ve got this covered: studies show that this valuation falls in range with that of most people writing on popular tech news websites! The difference is that these usually spend their days looking at press releases and lesser-known tech news websites and blogs written by even-lesser-known people (totally not the case here) to
repost find sources for their original pieces, while I usually spend my days going through computer engineering courses, building useless shit like Clouttery and answering tny.im support requests.
Since my 500-day-old posts have published, a lot of things have changed in the way I use Windows. Most importantly, my main Windows machine is no longer a Chinese Crapstore 7-inch tablet, but a proper Surface Pro 3 which I bought with a relatively good discount in October of 2015 right as the Pro 4 was being released. This means any problems I experience with Windows, I can no longer blame them on anyone other than me (the luser) and Microsoft (since the software is theirs, and the hardware is chosen, assembled and shipped by them). Oh, and drivers. On Windows, it’s always drivers.
After getting the Surface I started to use Windows much more, and in the last few months there have been many days during which I didn’t touch my more powerful Linux desktop . I mostly use the desktop for coding, compiling big code trees and running heavier programs, but I do all of my note-taking (OneNote!), light web browsing (redditting and hackernewsing, for instance) and ssh-ing into Linux servers using the SP3, which means that when there isn’t more than this to my day, I don’t even turn on the desktop. I bought the Type Cover 4 a few months after buying the SP3, which certainly contributes to how much I use it.
Still, and after these major changes to the way I use Windows and how often I use it, I thought it would be interesting to go through all of the complaints in my extremely popular post from 31st July 2015, the day Windows 10 was released to the general public as it (supposedly) went out of beta, and check what was done about each of them. In a completely unscientific way, obviously, matching the standards this publication has accustomed you to.
As with my older posts, I will focus on the desktop edition (i.e. x86, i.e. the version that can run Win32 apps, i.e. the version that is actual “Windows” and not Windows Phone or Mobile or whatever they call it this month). It was announced in September 2014, skipping Windows 9 for reasons that, after many theories, may be best described as “yes” . After multiple “preview” releases, it was made available “for real” on July 29, 2015 (this was version 1507, build 10.0.10240). Two days later, I published my “Windows 10 is unfinished” post, completely written and produced with my Windows 10-powered 7 inch tablet, sans-physical-keyboard. For this post, however, I’m not a masochist anymore – some parts have been written in my Linux desktop and others on the Surface with the type cover.
On November 12, 2015, the “November Update” was released (version 1511, build 10.0.10586), and I didn’t write a blog post because… well, I had more interesting things to do than I have right now (or rather, I was not procrastinating in writing). For the purposes of this post all you need to know is that this fixed a big number of stability and UI problems, but nothing too revolutionary. More recently, on August 2, 2016, the “Anniversary Update” was pushed to the production ring (version 1511, build 10.0.14393). This one was a bit more “revolutionary”, especially in terms of UI and feature changes, but as you’ll find out later… deep down, the important issues are yet to be solved.
There’s an upcoming “Creators Update”, but we are going to pretend that doesn’t exist and instead focus on the Anniversary Update, released about four months ago. Also, I can’t compare resource usage, as I’m no longer using the same hardware (even though I still own the smaller tablet and use it from time to time, it’s much, much less frequent now). Let’s begin.
The touch experience is still possibly worse than on Windows 8.1, but by now I’ve gotten used to it (also, with the Type Cover 4, I can use the excellent trackpad, so I no longer have the touchscreen as the only input device…). Annoyingly, the touch keyboard still doesn’t dock “properly”. This means that docking it doesn’t resize the whole desktop (forcing all windows to fit above the keyboard) like it did back in Windows 8.1. In many legacy apps which haven’t been adapted to “run away” from the touch keyboard, the cursor will still often be below the keyboard. Even on many parts of the Windows UI, this happens, and on parts which clearly have been adapted to avoid getting their inputs obscured by the keyboard, it’s still buggy as hell.
Funnily enough, some apps (like OneNote) can do this “desktop resizing” trick. So some months ago, I looked up the necessary APIs and in a few hours had a .NET proof-of-concept application that could resize the desktop, too. So it’s not a matter of missing APIs or compatibility problems – it’s certainly a deliberate design decision, and one I can’t understand. Perhaps it’s to encourage moving to UWP? I don’t even know anymore. If I still had to use the touch keyboard as my only keyboard for extended periods of time, you can be pretty sure that by now I’d have written my own “desktop resizing” helper that does its magic when the keyboard is docked.
The “void” between good old Win32 “classic” apps and the “modern” UWP (and that Windows 8 and 8.1 framework they don’t like to talk about anymore, but was the pre-UWP that powered Windows 8 apps) is even more reduced, with some bugs (for example, small details in behavior and looks between the “modern” and “classic” windows) getting fixed and more parts of the OS getting refreshed with a modern look (that, don’t let your eyes fool you, are often not actually built with UWP; it’s all just a matter of design language, the tech is still the same).
One can still find remnants of the multiple design languages used by the OS throughout its decades of history. Paying off this kind of technical debt, which I call “UX debt”, would surely go a long way towards giving Windows that polish it is often criticized for not having. You can still find icons from the Windows 98 and earlier era, from the XP era, from the Vista and 7 era, and even though it’s less noticeable (since it’s all “flat design”) you can also find some stuff that was designed back in the Zune and 8/8.1 days and was forgotten since then (now that I think of it, though, that happens mostly with Microsoft software other than what’s bundled with Windows). But pictures are so much better than text for this, so let’s mimic the iconic “two control panels” screenshot:
Not much left to comment here… oh wait, this is still a thing:
Sure, you may argue this is a fairly minor thing, doesn’t hurt anyone, doesn’t decrease system stability, and realistically doesn’t hurt anyone’s productivity. But, Microsoft, if you keep pushing your main product through deadlines without ever slowing down to fix “lower priority” stuff, the “UX debt” will keep increasing… damn, at some point you’ll lose even to “unpolished” Linux environments, if nothing else because some of these tend to throw everything away every three years and so have no “UX debt” to speak of (only bad UX, but no debt – it’s like being poor, but at least you don’t owe anyone money).
Now, the part which I personally find the funniest, and is an actual annoyance which certainly hurts productivity. To their credit, this had a major improvement with the Anniversary Update:
Left out of this image is the even wider assortment of context menu styles used by Microsoft’s own apps, like Office and Visual Studio. There are still different styles but the differences are now mostly between light and dark menu themes (which makes sense: dark elements produce dark menus). There’s also that giant menus situation, which supposedly should only occur when you use your fingers to open them. Problem: they sometimes show up when you use the mouse or a pen…
Don’t let me begin on the clusterfuck that are the network settings… I find myself constantly switching between the wireless networks popup in the taskbar, the “Network and Sharing Center” (old control panel page) and the Settings page to find the things I need. A particularly fun exercise I recommend to all readers, is getting to the dialog where you can retrieve the password for the Wi-Fi network to which you are currently connected. It’s especially fun if you have seen it before, and have a vague idea of where it is and how it looks like… and yet I always find it hard. Do you?
So, in terms of UI, it’s still far from perfect – but getting better. At this rate, I expect everything to be migrated to the “Windows 10 look and feel” in four or five years. Not a big problem, except for the fact that Microsoft, too, likes throwing out much of their work every three or four years (except they can’t throw much away, because of backwards compatibility and user training; things never get fully updated, and then you get “UX debt”). Let’s see how this works out.
There’s also the whole privacy concerns/telemetry topic, which I have skipped in this post. Not that it’s not important, but there’s so much to talk about that, it’d be better suited for whole another post on the privacy policies of most software-as-a-service (as some friends will be keen to point out, my own privacy policies – or the lack of them – included!). And the phenomenon of software that needlessly becomes software-as-a-service is vast enough to warrant another post, too… posts which I’ll most likely never write. But hey, just use your favorite search engine to find people with skeptical opinions on these topics. If you want a head-start and eventually some laughs, you can start with this or this. Guaranteed hours of endless fun and/or fumbling!
I also masterfully skipped over the “Windows updates are still stupid and require lengthy reboots 90% of the time! Why? Because it’s 2016 and Windows locks that way since the 90’s, that’s why!” subject, as well as the “Windows ate my settings and brought back Candy Crush!” one. Phew.
Now for the praising bit. Some of the stuff Microsoft is doing with Windows is particularly exciting. The Windows Subsystem for Linux, for instance: if it doesn’t appeal to you from an ethical/moral/religious perspective (and yes, I feel uncomfortable too), you have to agree it’s pretty interesting technically. And very recently they showed what appears to be a comeback of Windows RT, except this time it’s done right: x86 binaries running on ARM processors thanks to ISA translation. It’s not the fastest, and I suppose it only became usable (at least for “heavier” software like office suites and image editors) with the current generations of ARM processors (and perhaps more specifically, Snapdragon SoCs only). However, had they implemented this back in the Surface RT, perhaps not even as a marketing item but as a “nice to have” thing power users would find out about (just to find out it was dog slow), and the fate of that line could have been slightly better.
The Creators Update will bring more “nice to have” improvements and hopefully fix more of the still present UX issues, and add major features (but I’m not sure I care that much about these anymore, I would rather have more polished versions of what we have now). I bet: getting to the dialog where you “unhide” the wireless password will still be hard.
 This “desktop” is actually a Toshiba A660 laptop with a 1st-gen i7, 8 GB of RAM and a SSD upgrade, but the battery’s dead – it was never good to begin with. A machine that’s at least five years old and still rocking – unlike some would put it, it’s not ‘sad’ to be using it. Fortunately, I often can’t hear these people over the sounds travelling through the headphone jacks of my machines. I’m using Arch Linux, KDE and headphones and I’m quite happy with this setup, thank you.
 “Yes” is also the reason that’s more in line with the official reasoning, which, to quote Terry Myerson (an important guy at Microsoft), goes like this: “based on the product that’s coming, and just how different our approach will be overall, it wouldn’t be right to call it Windows 9”. Or to put it simply, “yes”.
The more base-2 oriented readers may be wondering, why post this 500 days after the release of Windows 10, missing the opportunity to post it 512 days after? The reason is that Windows 10 will turn 512 days old on the 24th of December, and you’ll probably have more interesting things to do that day than caring about this silly post – not that anyone cares on the other days of the year, anyway…
Because, why not? Let’s Encrypt makes it so easy…
Let’s Encrypt certificates are now used on all the websites maintained by Segvault, but not all of the websites of the TNY Network – the CPUVInf website, for example, seems to be using CloudFlare-provided TLS.
Go to the bottom, “Summing it up”, for the TL;DR.
The day I turn this website into a portfolio/CV-like thing will come sooner or later, and arguably that’s a better use for the domain gbl08ma.com than this blog with posts nobody cares about – except when I rant about new operating systems from Microsoft. But if you really care about such posts, do not worry: the blog will still exist, it just won’t be as prominent.
Meanwhile, and off-topic intro aside, the content usually seen on such presentation websites everyone-and-their-cat seems to have these days, will have to wait. In anticipation for that kind of stuff, let’s go in a kind of depressing journey through my eight years programming experience.
The beginning was what many people would consider a horror movie: programming in Visual Basic for Applications in Excel spreadsheets, or VBA for short. This is (or was, at the time; I have no idea how it is now) more or less a stripped down version of VB 6 that runs inside Microsoft Office and does not produce stand-alone executables. Everything lives inside Office documents.
It still exists – just press Alt+F11 in any Office window. Also, the designer has Windows 7 Basic window styles… on Windows 10, which supposedly ditched all that?
I was introduced to it by my father, who knows his way around Excel pretty well (much better than I will probably ever will, especially as I have little interest). My temporal memory is quite fuzzy and I don’t have file timestamps with me for checking, so I was either 9, 10 or 11 years old at the time, but I’m more inclined to think 9-10. I actually went quite far with it, developing a Excel-backed POS system with support for costumer- and operator-facing character LCD screens and, if I remember correctly, support for discounts and loyalty cards (or at least the beginnings of it).
Some of my favorite things I did with VBA, consisted in making it do things it was not really designed for, such as messing with random ActiveX controls and making it draw strange-looking windows (forms) and controls through convoluted Win32 API calls I’d have copied from some website. I did not have administrator rights to my computer at the time, so I couldn’t just install something better. And I doubt my Pentium III-powered computer, already ancient at the time (but which still works today), would keep up with a better IDE.
I shall try to read these backup CDs and DVDs one day, for a big trip down the memory lane.
Programming newb v2
When I was 11 or 12 I was given a new computer. Dual core Intel woo! This and 2GB RAM meant I could finally run virtual machines and so I was put on probation: I administered the virtual computers, and soon the real hardware followed (the fact that people were tired of answering Vista’s UAC prompts also helped, I think). My first encounter with Linux (and a bunch other more obscure OS I tried for fun) was around this time. (But it would take some years for me to stop using Windows primarily.)
Around this time, Microsoft released the Express (free) editions of VS 2008. I finally “upgraded” to VB.NET, woo! So many new things to learn! Much of my VBA code needed changes. VB.Net really is a better VB, and thank Microsoft for that, otherwise the VB trauma would be much worse and I would not be the programmer I am today. I learned much about the .NET framework and Visual Studio with VB.NET, knowledge that would be useful years later, as my more skilled self did more serious stuff in C#.
In VB.NET, I wrote many lines of mostly shoddy code. Much of that never saw the light of day, but there are some exceptions: multiple versions of Goona Browser made their way to the public. This was a dual-engine web browser with advanced UI, and futuristic concepts some major players copied, years later.
How things looked like, in good days (i.e. when it didn’t crash). Note the giant walls of broken English. I felt like “explain ALL the things”! And in case you noticed the watermark: yes, it was actually published to Softpedia.
If you search for it now, you can still find it, along with its website which I made mostly from scratch. All of this accompanied by my hilariously broken English, making the trip to the past worth its weight in laughs. Obviously I do not recommend installing the extremely buggy software, which, I found out recently, crashes on every launch but the first one.
Towards the later part of my VB.NET era, I also played a bit with C#. I had convinced myself I wanted to write an operating system, and at the time there was a project called COSMOS that allowed for writing (pretty limited) OS with C#… of course my “operating” systems were not much beyond a fancy command line prompt and help command. All of that is, too, stored in optical media, somewhere… and perhaps in the disk of said dual-core computer. I also studied and modified open source programs made in C# (such as the file downloader described in the Goona Browser screenshot) for my own amusement.
All this happened while I developed some static websites using Visual Web Developer Express as editor. You definitely don’t want to see those (mostly never published) websites, but they were detrimental to learning a fair bit of HTML and CSS. Before Web Developer I had also experimented with Dreamweaver 8 (yes, it was already old back then) and tried my hand at animation with Flash 8 (actually I had much more fun using it to disassemble existing SWFs).
At this point I was 13 or so, had my first contact with Linux more than done, through VMs and Live CDs, aaand it happened: Ubuntu became my main OS. Microsoft “jail” no more (if only I knew what a real jailed platform was at the time…). No more clunky .NET! I was fed up with the high RAM usage of Goona Browser, and bugs I was having a hard time debugging, due to the general code clumsiness.
How Ubuntu looked like when I first tried it. Good times. Canonical, what did you do?
For a couple of years, in terms of desktop development, I only made some Python scripts for my own amusement and played a very small bit with MonoDevelop every time I missed .NET. I also made a couple Lua scripts for Rockbox. I learned much about Linux usage and system maintenance as I used it more and more on my own computers and on my first Virtual Private Servers, which I got after much drama in the free web hosting communities. Ugh, how I hate CPanel.
It was around this time that g.ro.lt and n.irc.su appeared. g.ro.lt was a URL shortener that would later evolve into 4.l.to and later tny.im. n.irc.su was a social network built on Elgg, which obviously failed. I also made some smaller websites, like one that would take you to random image hosting websites, URL shorteners and pastebins, so you would not use the same service every time you urgently needed one. These represented my first experiences with PHP programming.
I have no pictures to show. The websites are long gone, not on the Internet Archive, and if I took screenshots, I have no idea where I put them. Ditto for the logos. I believe I still have the source code for the random-web-service website somewhere, at least the front page layout.
All this working on top of free stuff: free (and crappy) subdomains, free (and crappy) web hosting, free (and less crappy) virtual servers. It would take me some time until I finally convinced myself I needed to spend some money for better reliability, a gist of support and less community drama. And even then I would spend Bitcoin, which I earned back when it was really cheap, making the rounds of silly faucets and pulling money out of CPAlead-like offers through the use of multiple proxies (oh, the joy of having multiple VPS…). To this day I still don’t have a PayPal account.
This time, and when I actively developed tny.im (as opposed to just helping maintain it), was the peak of my gbl08ma-as-web-developer phase. As I entered and went through high school, I would get more and more away from HTML and friends (but not server maintenance), to embrace something completely different…
Low level, little resources: embedded systems
For high school math everyone had to use a graphing calculator. My math teacher recommended (out of any interest) Casio calculators because of their ease of use (and even excitedly mentioned, Casio leaflet in hand, the existence of a new and awesome color screen model that “did everything and some more”). And some days later I had said model in my hands, a Casio fx-CG 20, or Prizm, which had been released about a year before. The price difference from the earlier dot-matrix screen Casio calcs was too small to let the color screen go.
I was turning 15, or had just turned 15. I remember setting up the calculator and thinking, not much after, “I want to code for this thing”. Casio’s built-in Basic dialect is way too limited (and after having coded in “real” languages, Basic was silly). This was in September 2011; in March next year I would be releasing my first Prizm add-in, CGlock, a calculator PIN-locking software.
Minimalist look, yay! So much you don’t even notice it’s a color screen.
This was my first experience with C; I remember struggling with pointers, and getting lots of compilation warnings and errors, and run-time errors. Then at some point everything just “clicked in” and C soon became my main language. Alas, for developing native software for the Prizm, this is the only option (besides using C++ without most of its features, not even the “new” keyword).
The Prizm is a horrible platform, especially for newbie C programmers. You can’t use a debugger, nor look at memory contents, the OS malloc/free implementation has bugs (and the heap is incredibly small, compared to the stack) and there’s always that small chance some program damages your calculator, or at least corrupts your estimated files and notes. To this day, using valgrind and gdb on the desktop feels to me as science fiction made true. The use of alloca (stack allocation) ends up being preferred in relation to dynamic allocation, leading to awkward design decisions.
Example of all the information you can get about an error in a Prizm add-in. It’s up to you to go through your binary (and in some cases, disassemble the OS) to find out what these mean. Oh, the bug only manifests itself when compiling with optimizations and without symbols? Good luck…
There is a proprietary emulator, but it wasn’t designed for software development and can’t emulate certain things. At least it’s better than risking damage to expensive hardware. The SuperH-4 CPU runs at 58 MHz and add-ins have access to about 600 KiB of memory, which is definitely better than with classic z80-powered Texas Instruments calculators, but one still can’t afford memory- or CPU-intensive stuff. But what you gain in performance and screen resolution, you lose in control over the hardware and the OS, which still have lots of unknowns.
Programming for the Prizm taught me how it’s like to work without the help of the C standard libraries (or better, with the help of incomplete and buggy standard libraries), what a stack overflow looks like (when there’s no stack protection), how flash memories work, what DMA is, what MMUs do and how systems can be bricked when their only bootloader is not read-only. It taught me how compilers work from an end-user perspective, what kind of problems and advantages optimizations introduce, and what it’s like to develop parts of the C standard library.
It also taught me Casio support in Portugal (Ename) is pretty incompetent at fixing calculators, turning my CG 20 into a CG 10 and leaving two big capacitors out of a replacement main board. In this hardware topic, I learned quite a bit about digital logic from Prizm hardware discussions at Cemetech. And I had some contact with SH4 assembly and a glimpse into how to use IDA Pro. Thank you Casio for developing a system that works so well and yet is so broken in so many under-the-hood ways, and thank you Cemetech for briefly holding the Prizm higher than TI calcs.
I developed other add-ins, some from scratch and others as ports of existing PC software (such as Eigenmath). I still develop for the Prizm from time to time, but I have less and less motivation as the homebrew community has stagnated and I use my Prizm much less, as I went to university. Experience in obscure calculator platforms does not make for a nice CV.
Yes, in three years or so I went from the likes of Visual Studio to a platform where the only way to debug is to write text to the screen. I still like embedded and real-time programming a lot and have moved to programming more generic and well-known things such as the ESP8266.
Getting in the elevator
During the later part of high school (which I started in the fall of 2011 and ended in the summer of 2014), I did more serious Python stuff, namely Mersit, later deprecated in favor of Picored, which is not written in Python but in Go. Yes, I began trying higher-level stuff again (higher level, getting in the elevator… sorry, I’m bad at jokes).
My first contact with Go was when I was 17, because I wanted to develop something that ran without external dependencies (i.e., unlike Java or .NET) and compiled to native code. I wanted to avoid C/C++, but I wasn’t looking for “a better C” either, so Rust was not it. Seeing so much stuff about Go at Hacker News, one day I decided to try my hand at it and I like it quite a lot – I’m still unsure if I like it because of the language itself or because of the great libraries one can use with it, but I think both play an important role.
This summer I decided to give C# another chance and I’m quite impressed – turns out I like it much more than I thought. It may have something to do with trying it after learning proper languages vs. trying it when one only knows VB. I guess my VB.NET scars are healed. I also tried a bit of Java, in my first contact with it ever, and it seems my .NET hate converted into Android API hate.
Programming with grades
University gave the opportunity (or better, the obligation) of having other people criticize my code. The general public could already see the open-source C code of my Casio Prizm add-ins, and even the ugly code of Goona Browser, but this time my code was getting graded. It went better than I initially thought – I guess the years of experience programming in different languages helped, especially as many of the people I’m being compared with have only started programming this year.
In the first semester we took an introductory programming course, which used Python, and while it was quite easy for me, I took the opportunity to learn Python to a greater depth than “language in which to write quick and dirty glue code”. You see, until then I had not used classes in my Python code, for example. (This only goes to show Python is a versatile language, even if slow.)
We also took an introductory computer architecture course where we learned how basic CPUs work (it was good for gluing all the separate knowledge I already had about it) and programmed in assembly for a course-specifc CISC-like architecture. My previous experience with reading SH4 assembly proved quite useful (and it seems that nowadays the line between RISC and CISC is more blurred than ever).
In the second semester, I had the opportunity to exercise my C knowledge, this time not limited to the Prizm platform. More interestingly, logic programming, a paradigm I had no intention of ever programming in, was presented to us. So Prolog it was. It went much better than I anticipated, but as most other people who (are forced to) learn it, I have no real use for it. So the knowledge is there, waiting for The Right Problems(tm). I am afraid I’ll forget much of it before it becomes useful, but if there’s something picking C# up again taught me, is that I can pick up pretty fast skills learned and abandoned long ago.
The second year is about to begin and there’s some object-oriented programming coming, I hope I do well.
Summing it up
I do not like “years of experience” as a way to measure language proficiency, especially when such languages are learned for use in short-lived side projects, so here’s a list with an approximate number of lines of code I have written in each language.
- C: anywhere between 40K lines and 50K lines. Call it three years experience if you will. Most of these were for Prizm add-ins, and have since been rewritten or heavily optimized. This is changing as I develop less and less for the Prizm.
- PHP: over 15K lines, two years if you want to think that way. The biggest chunk of these were for developing the additions to YOURLS used in tny.im, but every other small project takes its own 200-500 lines of code. Unfortunately, most of this is “bad” code, far from idiomatic. The usual PHP mess, you know.
- Python: at least 5K lines over what amounts to about six months. Of these, most of the “clean” lines (25-35%) were for university projects.
- Go: around 7K lines, six months. Not exactly idiomatic code, but it’s clean and works well.
- VBA: uh, perhaps 3 or 4K lines, all bad code 🙂
- VB.NET: 10K lines or so, most of it shoddy code with lots of Try…Catch to “fix” the problems. Call it two years experience.
- C#: 10K lines of mostly clean and documented code. One month or so 🙂
- Lua: mostly small glue scripts for my own amusement, plus some more lines for use in games such as Minetest, I estimate 3-4 K lines of varying quality.
- Java: I just started, and mostly ported C# code… uh, one week and 1.5K lines?
- HTML, CSS and JS: my experience with JS doesn’t go much beyond what’s needed to modify DOM elements and make simple AJAX requests. I’ve made the frontend for over 5 websites, using the Bootstrap and INK frameworks.
- Prolog: a single university assignment, ~250 lines or one month. A++ impression, would repeat – I just don’t see what for.
In addition to all this, I have some experience launching the programs and services I make – designing logos/branding, versioning, keeping changelogs, update instructions, publishing, advertising, user support. Note that I didn’t say I’m good at any of these things, only that I have experience doing them, for better or worse…
Things I’d like to have more experience with:
- Continuous integration / testing in general;
- Debugging code outside of .NET/Visual Studio and printing debug lines in C;
- Using Git and other VCS in big repos/repos with more people (I want to see those merge conflicts and commits to the wrong branch coming);
- Server-side web development on something other than PHP and Go. And learning to use MVC frameworks, independently of the language;
- C++ (and Java, out of necessity. Damned Android);
- Game development. Actually, this is how many people start, but I’m so cool that I started by developing POS software 🙂
After yesterday’s popular post Windows 10 is unfinished, where I bashed said OS, today, I’m going to praise Windows 10 (where possible). This is so we can keep with the opinion diversity people are now accustomed to seeing on the Web, faithfully satisfying the thousands of Reddit and Hacker News users who can’t skip a beat on hot technology topics and especially, hot discussions on those topics.
A lot of people took my post as my definitive opinion on the matter and also as if I was telling some universal truths, and mistakenly concluded that I only had negative things to say about Microsoft’s latest big release. Others were saying I focused on the wrong problems; that the design issues were minor nitpicks, and effectively they are, when compared to the functionality problems (which I’m also having, but apparently that part was overlooked). My intention was not to write a fanboy post nor to start flamewars, and that’s the case with this post too.
Yesterday’s post was written from start to end on my Windows 10 tablet, without hardware keyboard (yes, it was painful, but not as much as it would have if using an Android tablet with similar characteristics), including screenshots and image editing (MS Paint FTW!). That’s not the case with today’s post, that was written with my laptop, because Microsoft is yet to issue an update to fix the virtual keyboard in Windows 10. The OS it is running doesn’t matter; let’s just say I’m writing this in MS-DOS 6.0’s edit.
Let the deserved Windows 10 appraisal start.
I upgraded from Windows 8.1, before Microsoft decided it was ready for me to install it. Yes, I forced the download and installation process. I wanted to get it downloaded before the end of July, so that it would not count towards this month’s data cap. I wanted to get it installed because I thought it would have tons of updates to download in the first days (not the case), and also because I’m going to need this tablet operational by September when university classes begin, so I thought I better get used to it and point out all mistakes sooner rather than later.
Yes, I could have stayed for another year on 8.1 before losing the option to upgrade for free, but I’m also interested in developing Universal Apps, so here’s that.
Despite me rushing the update and the tablet having 32 GB of storage of which only 22 GB are for the Windows partition, the process went perfectly, and apparently I still have the option to go back to 8.1 if I wish (at the expense of only having 2 GB of free disk space on C:). All data and apps were kept, except f.lux, possibly because (as far as I could understand when uninstalling its remnants) it was installed in AppData (note that AppData is mostly kept, too, but f.lux in particular wasn’t).
From leaving Windows 8.1 to seeing Windows 10 desktop it took my tablet about a hour and half. The flash storage on it is not especially fast (definitely not a SSD), which probably explains why most people can do it in one hour.
All points taken into account, the upgrade process went surprisingly well and was fast, as appears to be the case with the majority of users. Much better than ending up with a system that doesn’t boot at all, or with driver issues (which some users are still having), which as far as I remember were popular problems in previous versions’ in-place upgrades. Also, kudos to Microsoft for making it work on devices with such a limited amount of system storage.
There was the first-run setup, where the polemic privacy defaults are located (I disabled almost everything), but the most complicated part is what comes when the system finishes installing. In my case, Windows understood this was a tablet and accordingly selected tablet mode automatically. Because on 8.1 I basically only used the desktop, and because I thought it would be easier to find most settings on the desktop mode, I immediately went looking for the switch and since then I have only used desktop mode.
The desktop mode still works very well with touch screens; I have gone back to tablet mode for five minutes just to check it out, but went back quite fast, as I deemed the desktop good enough. Tablet mode didn’t fix the problem of the touch keyboard appearing over other windows even when docked, which would have been its major selling point for me right now.
Windows 8’s modern apps were kept from the previous version, including the MSN-powered apps such as Travel, which have been discontinued and will stop working in September. Of course, those who have an Universal app replacement (Mail, Calendar, Twitter, Maps, possibly more) are replaced. In the case of Mail and Calendar, it remembered the previously added account, but I had to pair them again in the case of Google and Microsoft accounts, and re-insert credentials for IMAP accounts.
OneDrive apparently now refuses to have its folder out of the C: drive, or perhaps that’s only a problem when the folder you want to chose is on a removable drive. I solved this problem by mounting the SD card, where I had the OneDrive folder, on the C: drive (NTFS mountpoints FTW!), then pointing OneDrive to this mountpoint. Yes, I know what I’m doing and you should too. This SD card, unlike what Windows thinks, is never removed.
I also had to download desktop Skype. Before I was using the Modern UI version of Skype, which was discontinued some time ago. But the desktop version uses so much RAM and is less touchscreen friendly, making it one of the most annoying parts of my Windows 10 experience. It also doesn’t update with new messages during Connected Standby, which is a thing my tablet has and I’m going to talk about later, and it doesn’t put its notifications in the new Action Center, either.
People are saying the tablet experience has actually gone worse with Windows 10, but to be honest if they fixed the touch keyboard I’d say it is as good as Windows 8. Of course, if you are used to the charms bar and to the gesture of “swiping down an app” to close it, you’ll be out of luck:
- swiping from the top on a window does nothing except move or restore it (if it was maximized);
- swiping from the left opens the Action Center (where some handy, more or less configurable shortcuts are located, so you won’t miss the “Settings” part of the charms bar);
- swiping from the right shows the task view, where you can switch apps and desktops;
- sadly there’s no longer a way to bring up a big clock, even when running full-screen stuff (games, videos…), something the charms bar was good for.
As it’s been widely reported, now Universal apps, Windows 8 apps and “normal” software made for the Win32 API all work together, with the same window borders and titles and showing on the same task lists. If only it had been this way since the beginning, Windows 8 would not have received so much negative critique and “Modern apps” could have actually been more used. Yes, I believe windows are adequate even for tablet devices (and not just by putting two windows side-by-side), and that is certainly one of Windows differentiating factors in the world of tablet OS.
I still can’t comment much on this part, because I’m having some issues with my Voyo A1 Mini that look not like Windows fault but driver problems. The “System” process (i.e., the NT kernel) is often using multiple MBs of RAM. I know I’m not the only user with this problem; there is at least one known bad network driver, but I don’t use it. I’ve also seen suggestions for disabling the network device usage service, but in my case that didn’t help. The result is that it always has 90-95% of physical memory used, and the commit charge at something like 3 GB of 3,9 GB.
I have also noticed search indexing stuff has gone more aggressive again on Windows 10, after being mostly quiet on 8.1 (as far as I could see). But since I haven’t done any serious monitoring, this could be just my impression.
The update could also have damaged the special CPU throttling set up for this device, given that it now runs much more hot than before, even for the same typical load. It appears the CPU (Intel Baytrail) works at higher frequencies more often – just a slight load and there it goes to 1,55 GHz or so (the “announced speed” of the CPU is 1,33 GHz). I have updated to the latest DPTF (Intel’s thermal stuff) drivers and it reduced the problem a bit, but it’s still present.
Now, this isn’t all that bad, given that Windows is very responsive even with the CPU at 75 degrees Celsius and 95% of the physical memory used. Let’s just wait for updates, both for Windows and for drivers, before taking more conclusions.
Connected Standby is still annoying
My tablet supports Connected Standby. On Windows 8, it was more or less like suspending the computer, but Windows Store apps could still run in the background to perform small tasks, and if you were playing media in such an app, it would keep playing even with the screen off – just like with Android devices.
The problem is if you want to use something other than a Windows Store app (read: 99,9% of the software available for Windows) to play music, or download files, or if you want to watch YouTube with something other than IE’s Modern UI mode. Windows will just suspend desktop apps and they will stop playing, or downloading, or crunching numbers. What makes this really annoying is that there is no way to turn off the screen without entering Connected Standby. So it’s burning extra battery and, at night, our eyes too.
In Windows 10, Connected Standby is more or less the same thing. I hoped that with Windows 10 they would add an option to be able to white-list certain “old fashioned” (Win32) apps into running during connected standby, or alternatively, a way to turn off the screen without going into standby.
At least, the “Sleep” and “Turn off the screen” settings now seem a bit better decoupled, and with my current settings (turn off screen after 2 minutes, sleep after 4) there is a bigger delay between when the screen turns off and the music stops playing. During this delay one can tap the screen and it will turn back on, instantly. Just like with a normal laptop that turns off the screen after a while. Let’s just hope Microsoft doesn’t consider this to be a bug and doesn’t “fix” it.
I can’t comment much on the Cortana feature itself, but I can comment on the stuff surrounding Cortana and whether the feature is enabled or not. Here, Windows is set up with a system language of US English. The region was set to Portugal, and the time and date and formatting settings to Portuguese. I was told by a friend I had to set my region to US for Cortana to become available, and that’s indeed true.
I just don’t understand, if Cortana is going to speak in English anyway (because that’s the system language), why does it have anything to do with the region. Unless it is expecting to change the language it uses depending on the region setting, and not depending on the language I want to see (and hear) stuff in. Oh well.
Microsoft seems really interested in listening to what the users have to say, so there’s a dedicated feedback app and everything. Unfortunately, this app filters content by region instead of filtering by language, which limits what reviews you can see and upvote. I wonder if anyone from Microsoft will look at the feedback of less populous countries like the one I live in, and even smaller ones.
Microsoft also seems really interested in learning how people use the OS, so much that only Enterprise users can completely disable this kind of feedback. Privacy concerns aside, I really hope the data generated with these feedback tools won’t be used as motivator or justification for taking away even more features and customization ability.
I always wanted to move to a rolling release Linux distro, but I’m yet to make the move; it appears I switched to a rolling Windows release before I did the same with Linux! I actually think it is a very good idea to stop releasing major versions and put new things out in a more continuous way. Major upgrades are a hassle, even when the upgrading itself takes just one hour – first, a giant download, then having to wait while the Windws upgrades and reboots multiple times, then having to set so many little settings that are new or changed with the new version…
I would be even happier if every user had the ability to refuse or at least delay certain updates (even because of, say, known driver and software incompatibility issues). The way things are done right now, only makes the whole thing look like a giant Microsoft-controlled botnet and by paving the way to Windows-as-a-service, makes people fear a future where you’ll pay for Windows by the month (and perhaps by the window/app/user?).
Finally, it’s about time Microsoft finds an ingenious way around the way file handles work in Windows, such that system files can be replaced without rebooting the system. Or at least, they could make the reboots less disrupting, for example by “suspending” the apps before the reboot, then restoring them.
My conclusion is to sit and wait. Windows 10 is actually pretty good for what feels like the end result of a development cycle damaged by setting a release date way too early. It should have been ready when it was ready, but I understand Microsoft not wanting to deal with another “XP to Vista” situation, where it took five years to release a new OS version with an abandoned revolutionary version in between, and a shitty end result. This way, the most people can say is that it’s shitty, but at least it came on time.
If you are using Windows 7 on a desktop and are happy with it, or using 8.1 on a tablet, I don’t think you have much to gain by upgrading now, unless you desperately want to use Cortana. People using Windows 8.1 without a touchscreen may find more value in upgrading now, especially if they use Modern UI apps and are annoyed by the context switches between them and the desktop.
Anyway, I always wanted to try Longhorn in its unstable and unpolished state, and now here is an opportunity – not with Longhorn, but with another revolutionary Windows version that while stable, has its own big polishing needs. But we already talked about that…