Podio / Geeks Who Drink Collapse

We’ve been supporting Geeks Who Drink for a few years now. They have a Java application that helps them publish, distribute, get paid for, and pay quizmasters. Noventum maintains this application, so they can keep helping people have fun at pubs, bars, restaurants, and now Dave and Buster’s! They are generally a fun client to work with, and try to add positivity to the world through their products.

One thing that is not fun is when a software system your software system relies on breaks. That’s what happened when their vendor, Podio, crossed a boundary around June 29, 2022. This boundary isn’t something most non-programmers think about too often. 

As an example, suppose you want to buy this padlock from Amazon:

Master is a solid lock brand, and everything seems OK with it, AS LONG AS YOUR COMBINATION IS BETWEEN 0-9999. If you really want to set your combination to 8675309, this lock is not for you, and you’ll have problems. It will “overflow” the potential digits. The same with an odometer in an old car – it will not track more than 999,999 miles on a car.

Software systems have the same limitation. Instead of digits on a lock, or odometer, they have space in bits. A “bit” is a one or zero. Here’s an example for three bits:

This same pattern continues:

This follows the 2^n -1 pattern, where “n” is the number of bits you have.

The Java programming language stores integers (a type of number) in 32 bits. The maximum value for an integer in Java is (2^32 – 1) / 2 = 2147483647. I slipped in the divide by two, since the Java language wants to actually allow negative numbers, so the range is -2147483647 to 2147483647.

Our client relies on a system called Podio, that allows for very easy form and (sort of?) application development. A lot of their data is stored in Podio, and then Podio passes that data to their custom application (which Noventum works on.) Think about putting a bunch of data into a spreadsheet, and having that online spreadsheet feed the data, automatically, into another program, and you’ve got an OK idea of how Podio works.

Around June 30, Podio exceeded 2147483647 for the ID values it was passing into the Geeks Who Drink application. So the Geeks system couldn’t handle any new information coming from Podio. In addition, Podio provides a library, which they no longer support. Our application uses that library to talk with Podio.

So, what did Noventum do? Initially we had a fairly fruitless support exchange with Podio, in which they confirmed our issue existed and was real, and did nothing else. Then within the first day, we realized we needed to get to work, and fix the problem ourselves, since we’d be getting nothing from Podio. Brian stopped being a manager and CEO, and rewrote their Podio library, and modified the application, and Rajesh fixed a LARGE number of automatic tests, like 610, and we got everything working. We replaced the integer data type with a “long” data type, that has a maximum value of 9,223,372,036,854,775,807. So this should be good for a while… The entire process took about two weeks. The most stressful portion was the first week, when we weren’t sure if our approach would work, and no new data flowed into the Geeks Who Drink system from Podio. After some initial successes, and it became clear the approach we were taking would work, the stress level decreased substantially, even if the effort required was still substantial. The tests helped us feel confident that the code change was correct.

Out of the 1102 source files for this project, we modified 239 of them (21.68%) with our initial change, and maybe another 30 with subsequent smaller fixes.

The only reason this was possible was because Podio provided the source code for their library, for anyone to modify. This is a concept called “open source.” Without this, it would be basically like a locked hood on a car, so no one except the manufacturer could open it up and work on the engine. We like open source at Noventum, and work with it exclusively.

Podio also practices a concept called vendor lock-in. We cannot export chunks of Podio’s online system, and run them ourselves. We are dependent on Podio, and have no control over the fact that their IDs exceed two billion. I believe their library provided a good balance between corporate control and ownership, and hippy dippy open source everyone owns everything as a communist. They still control their library (neglectfully) but the open source nature allows others to fix it, when there are problems. This is not possible with their web solution – they completely control that, and do not provide the source code for you to run and manage your own Podio.

Thankfully, Geeks Who Drink could continue to use the system, as-is. They couldn’t add new employees, or venues, but all the existing people made sure quizzes happened as expected. The new places still had quizzes, but all of the administrative details associated with that needed to be run by hand, as opposed to by software. Double thankfully everyone stayed calm during the entire process.

The lesson from this fix is to be careful where control and ownership lies in your software products, and dependencies. The “no code” solutions provided by vendors may seem very attractive, at first. Later on, you may regret the fact that you have no control over this “inexpensive” software. This is the third time Noventum has been involved in such a project (for two other clients), and the risks associated with using vendor controlled software need to be balanced against the gains provided by using someone else’s systems. We are pushing to completely remove this dependency on Podio.

* Geeks Who Drink was not charged for the time in writing this article. Just the time fixing their problem.