I don’t know about you, but as someone born in the early 1990’s, I found Neopets to be one of the more interesting things on the internet in the early 2000’s. Created in 1999, this virtual pet website was a complex web of Flash games, commerce, and daily adventures. There was always enough to keep kids busy for hours.
Growing up with a keen interest in anything that used electricity, I was naturally interested at the time in the technology that powered Neopets. I wouldn’t be the only person to say their first experience ‘coding’ was making their Neopets shop look fancy (the site allowed basic HTML markup to make your own store look the part).
Not only was the site a fun internet playground at the time, but it was also full of wit and useful life advice:
The website used Flash extensively for all games and maps (maps were the main way to navigate around all the different ‘worlds’), but the rest was server-side rendered HTML, with a bit of Javascript thrown in too, basically all powered by PHP. Given the timeline of PHP, the site would’ve started off using PHP3 in 1999, but probably would’ve moved to PHP 4 when that was available in 2000. This is all speculation, of course – it could still use PHP3 for all I know!
Actually, nope, HTTP requests are telling me they are still on PHP 5.4.28 (released 1st May 2014). I’m sure they wouldn’t be running an insecure version though. Oh wait. At least Register Globals had been removed by this point in PHP’s lifecycle.
I can’t imagine what atrocities lie under the hood. The site was subject of a data breach in 2013. Every page is a different .phtml file (that extension alone is a throwback). Browsing the HTML source reveals all sorts of things, such as tonnes of ad-hoc inline Javascript on various pages, the distinct lack of form nonces, and the short <? syntax for echoing data (not a default option in PHP for quite some time). I even remember once accidentally uncovering an SQL injection issue when trying to fill out a form.
The site is still alive, although the demise of Flash has made it very difficult to use these days. Similar Web estimates the site has had 4.22 million visits in the last six months, with a whopping 58 average pages per session, which is nothing too scoff at. Pretty impressive, actually, given I personally can’t work out how to make the Flash components on the site load at all.
An average of 34,555 visitors per day is some good traffic. Given an average of 58.41 pages per visit, this equates to 2,018,357 page views per day. The site has a couple of banner ad units on each page, so if we assume a very modest eRPM (effective revenue per thousand pageviews) of $1, that is $2000 per day revenue. The eCPM may be even lower, given the audience of children and the advertising restrictions that come with that segment, and the banner-blindness that would come with users being on the site so much – but still, there’s a good chance Neopets is still bringing in some good ad revenue to keep the business running.
And then there is Neopets Premium, a subscription membership service to unlock additional site features – the cheapest plan being US$5.83 per month.
Yet, in 2020, the new Chinese owners are trying to bring the site into at least the 2010’s, with a new responsive, mobile-friendly version of the site. And what a task ahead they have – by my estimate, there is something like 300 Flash games on the site, dozens of Flash maps, and hundreds upon hundreds of HTML/PHP pages. All of these will need converting.
And personally, I am intrigued that 2020 is the year they start launching this work. I just assumed the site was on life support, ready to die off. But no, just a few weeks ago I got an email boasting the new beta version, now available for anyone to try. I couldn’t resist logging back in, and taking a look.
Flash Player will be completely unavailable for download by 2021, so it make sense that they should try and upgrade the site quick fast to avoid killing it. Preferably 5 years ago…
So first up, this site is web-first. They haven’t decided to rebuild the site as a mobile app. They are literally going through pages, one by one, and rebuilding them with a new design that takes advantages of novel technologies, such as media queries.
I’m happy Neopets is sticking to their heritage, and getting the web right. This allows them to leverage a lot of their existing code and architecture, and make it playable on desktops and mobiles. A complete rewrite would’ve been foolish.
Only a few pages and features have been converted yet – clicking most of the menu items warns me I’m about to head back into the unfriendly wilderness.
But of the pages that have been converted, there is a bit to unpack. Firstly, let’s talk framework and approach on a static page such as the homepage.
Frameworks… so this is no Single-Page Application (SPA), or anything fancy like that. A quick ‘view source’ reveals we have jQuery, and lots of vanilla Javascript too. Some Javascript is in separate files, but lots of just sprinkled throughout the page.
I’m sure there’s a tonne of legacy considerations with this codebase, so I don’t want to knock the engineering effort here, but it’s clearly a mess. I’m sure the HTML, CSS & Javascript we see here is only a fraction of messy code that has accumulated over the 21 year history of this site.
Everything new appears to include ‘2020’ somewhere in the naming convention (a decision I’m sure will get annoying for any engineer as soon as the clock ticks over to 2021).
There are all sorts of comments to be found browsing through the scripts. For example, here we can see this bit of code has been moved to a PHP include file:
There was also an attempt at resizing games (more on games later…):
Let’s jump across to a map and start exploring. Now, I can’t find a way to directly get to the one functional map from the main menu, but the Release Notes have a handy list of converted pages so I can find them.
The one converted map is of Mystery Island. Already the font and margin on the text is bugging me, but this is a beta, so I’ll cut them some slack here. What I’m really interested in is how they have re-implemented the map.
My initial feeling was they would’ve written it as a HTML5 Canvas, or maybe just gone low-tech image maps as a quick-and-dirty solution here (some areas of Neopets already have image map versions, although they are hard to find these days).
And my initial reaction of a HTML5 Canvas map is correct. Kind of.
Remembering how the site used to use Flash extensively? Well, it seems this new approach for maps leverages this old technology.
However, a quick 10 second something search reveals you can import the old Flash projects into Adobe Animate, and then export them to HTML5 Canvas. I didn’t realise this was a thing, so I quickly downloaded Adobe Animate, found a SWF file on the Neopets site, and dragged in a SWF file to Animate.
Lo and behold, I got the map!
Now, this approach didn’t allow me to access the ActionScript, or even view the animation or links – so it’s not as if I can chip in and help upgrade these components myself. Indeed, even if I did get access to the original scripts, it seems some manual work is still required in areas to help with display scaling, keyboard/mouse inputs, etc. from within a canvas.
This approach to porting maps seems sensible, given how many Flash resources Neopets would’ve generated over the years. There’s no point recreating them by hand if there’s an easier way.
This brings us onto games…
At the time of writing, seven glorious games have been ported (out of an estimated 300).
Now the games in themselves will require a Herculean effort to port, if only because of the quantity of them. Thankfully, not many of them are very complex. Most games I remember really only have one level of gameplay, and just progressively get faster/harder as you earn more points.
These seven games thus far are all Adobe Animate exports. All the code has been obfuscated, and there are loads of binary blobs for images and sounds.
The part I’m most interested to understand is the score submission process. Points in games translate to in-world currency, with a ratio based on how hard the game is. Scores and anti-cheating measures have always been difficult, and from early on the Flash games had four anti-cheating mechanisms in play:
- You can only submit scores from each game a maximum three times in a day
- You can only score a maximum 1000 Neopoints per game play
- High scores must be manually approved
- The score submission takes place in Flash, with a token request/response style system to try and obfuscate the HTTP submission requests. This has been packet-sniffed and cracked multiple times over the years
Three of these measures are easy to enforce server-side, but this last measure will get very difficult when everything is Javascript only.
Currently, score submission from the HTML5 games is blocked. You can play, but can’t get any in-world currency.
The score submission JS (which currently fails), is right there, albeit obfuscated. This payload could probably be reverse engineered fairly trivially.
Neopets probably have a couple of server-side measures planned to help avoid cheating once they turn on score submission again:
- Compare time played vs points scores, and block any unrealistic submissions
- Block users who send too many malformed requests, to try and catch casual people mucking around in their Web Inspector tools
In addition to this, they could also make the browser send the game actions second-by-second, and calculate scores server-side. This is probably too big a project for these ~300 games, so I guess they’ll just rely on some easier measures initially.
Besides, even if they did block all forms of unrealistic score submission, it would still be possible to automate gameplay with this browser.
I do hope some of the games get little upgrades as they are ported across. For example, I’d love to see Advert Attack get upgraded with some Cookie Consent dialogs, instead of the popup ads. That would be a nice touch.
Outside of the Flash porting, they have also ported one lonely shop. It’s just the standard grid, resized to fix on smaller screens.
Given the significant role of commerce on this site, I’m unsure why they haven’t ported more shops. They all follow the same basic template. Even if the maps to get to the shops didn’t exist, the basic UI of the Shop Wizard would’ve helped us navigate and buy a much larger range of products.
All in all, it’s good to see this site getting a new lease on life. There isn’t a whole heap to do yet (there’s only so many times I can play Hassee Bounce without redeeming the Neopoints), so maybe in a few months I’ll have to check back and see how they’re doing.