ZOMG, Piwik is HUGE

So, last week-end was the big move: I moved Wiki4Games.com, Patheticcockroach.com, and most of subdomains (notably the notepad), to the new cloud server. I also migrated the BBS from MyBB to SMF (Simple Machines Forums). Glad to be finally rid of that awful thing, SMF is maybe surely not as fancy, but has all the tools needed to fight spam, and a fairly simple plugin system (about that, maybe I think I should post that private (sub)forum plugin I patched some time). Anyway, I didn’t have the time to move everything that week-end, and I thought I’d just finish the remaining small things some evening in the week.
So, yesterday I found myself getting ready to transfer the Piwik stats tracking system. And I discovered that its database had grown to 180 MiB! Ok, it’s not exactly big (for example Wikpedia’s database is, just for the pages, around 5TiB), but when you consider that this corresponds only to 6 months of statistics, with around 1000 unique visitors a day, this sounds a bit too much. Plus that’s actually almost 3 times the size of Wiki4Games’s database…
Piwik are totally aware of that issue, so I hope they’ll eventually find a way to deal with it. I’m aware that’s not a trivial problem, but that’s one that needs to be taken care of, or the thing will just end up unusable. For instance if we had 10k visitors a day on Wiki4Games, this would result in 3 GiB a year just for the stats… that sure can be dealt with, but wouldn’t be worth the money spent on it. I wouldn’t mind keeping only a few months of detailed stats and then only basic daily numbers for older periods, and I think I’m not the only one, so probably a quick’n dirty first fix could be just an option to drop old detailed data. Well, time will tell.

Edit: tickets 🙂
http://dev.piwik.org/trac/ticket/53
http://dev.piwik.org/trac/ticket/5

Re-edit: As of today (version 1.4) the big fat database issue growing like mad still isn’t solved and seems of fairly low priority to them. In the meantime my database grew to 250 MiB… Also, for some reason every time I leave a comment on their blog it’s not published, so I won’t be posting any more things about them. Although I wouldn’t go as far as to recommend Google Analytics over them, I’d encourage you to find an alternative solution developed by less narrow-minded people caring a bit more about your disk space. Also, don’t forget that you can block most stat-tracking software (including Piwik) using Ghostery.

Another update (19 June 2011): Piwik 1.5 is being released (they have a blog post about it but it’s not yet available through auto-update) and final features an automatic log-pruning option that should fix those database bigness issue. Note that the auto-update feature checks for a new version every X hours (probably every 8 hours?): if you want to force a version recheck to skip the wait, go to your Piwik database, then in the [your_db_prefix]option table, then edit the value of UpdateCheck_LastTimeChecked (for instance set it to 1: this corresponds to the timestamp of the time where the latest update check was run). Then reload your dashboard and you’ll have the new version notification, if any.