Skip to content


Weight, sport and health

Well, a post not at all related to informatics for a change. Basically just a couple of bookmarks I really didn’t want to lose:

First the most interesting one:
Obesity and Mortality Risk: new findings from body mass index trajectories: Medscape‏ (restricted access), PubMed entry, direct link to the article on the journal’s site. To summarize this very briefly, in the studied population (US adults aged 51 to 77 years), the highest survival rate was among overweight people with a stable weight. Overweight with a small weight gain followed. Normal weight (with weight gain) was third. Obese was the worst. The results were robust when adjusted on baseline demographic and socioeconomic characteristics, smoking status, limitations in activities of daily living, various chronic illnesses, and self-rated health. Long story short, when you get older, it seems better to have a BMI a bit above the “limit”. Could also be just a sign that this group of people with a higher BMI but not obese are both fit and not dieting themselves into starvation 😉 I’m fit and above 25 myself already.

Too Much Sport for Teenagers May Be as Bad as Too Little: Medscape‏ (restricted access) or similar article on MedlinePlus (free access) or WebMD (free access, seems to be a copycat but on a better designed site ^^). Questionnaire given to 1200 Swiss teenagers between 16 and 20 year old, asking for physical activity level and a questionnaire on quality of life. The low activity group (< =3.5h of physical activity per week) and the very high activity group (>17.5h/week) were more than twice as likely as those in the average group (3.6 to 10.5h/week) to score below 13 on the well-being scale. Those in the high-activity group (10.6 to 17.5h/week) had about half the risk for poor well-being as those in the average group, with the peak score of well-being occurring around 14h/week. I didn’t read the full article, the most crazy thing about this in my opinion is the amount of sports those guys do. I’ve never done more than maybe 7h/week – and even this seemed higher than everyone else I know -, unless you include all the walking (which I don’t think was the case here).

Bonus (2014-01-15): DASH diet plan there, recently rated best overall diet in US News & World Report: “Find the Best Diet For You.”.

Posted in health & medicine.


How to install MJ12node (the Majestic-12 distributed crawler) on Debian 7 x64

Get the package prepared by refic here: https://www.majestic12.co.uk/forum/viewtopic.php?t=4423
As of today (22 December 2013), the latest appropriate version (LEGACY) can be obtained directly using:
wget http://mono.mj12.fi/mj12node-bundle/mj12node-linux-v1710-bundle-32bit-mono-3.2.1-LEGACY.tgz

Then you need to install “some” (there’s quite a bunch of them actually, better run this in screen if you’re on a remote connection) 32 bits libraries:
(NB: unlike the other parts which should be run as the user dedicated to MJ12node, this part needs to be run as root/sudo)
dpkg --add-architecture i386
apt-get update
apt-get install ia32-libs

And as of Debian 7 that should be all (I think before that you also had to install ia32-libs-gtk, so if you are on an earlier version and the above isn’t enough for your node to work, you can try adding this package too). Now back to the archive you downloaded: unpack it then run it:
tar xf mj12node-linux-v1710-bundle-32bit-mono-3.2.1-LEGACY.tgz
cd MJ12node
./run.sh

Now this is just a first run. If everything works, press ‘s’ to activate the webserver control interface. Then access your machine in your browser on port 1088 (http://your_machine_s_IP:1088), and configure it, notably enter your account information in the option panel and don’t forget to save. At the moment there’s a bug in the software: once you’ve entered your login details, it won’t be able to fetch work until the node is restart it. So go to tools and restart your node. Or, if you don’t want to wait the standard 30 seconds imposed for a restart, shut it down and then restart it manually (./run.sh in console again, once the node did shut down). This second option is my favorite, notably because it’s the occasion to relaunch the node via screen this time (it’s not a daemon so you need to run it in screen).

Well, that’s about it. If you don’t know about screen, here are the commands:
apt-get install screen (shouldn’t be needed, but in some light distributions it’s missing!)
screen
./run.sh

Another remark: the web interface can be manually opened and closed from the console by pressing respectively ‘s’ and ‘t’. You can also configure the node to always run the web interface, with a password, on a custom port, etc., via the options.

Posted in servers, software.


Tips to fixing a broken RAID 1

As often, I don’t really have the time to turn this into a proper guide. Plus some others do exist already, such as Replacing A Failed Hard Drive In A Software RAID1 Array. So I’ll just list some useful commands to diagnose a broken RAID and fix it in the case where there is no physical issue on the disk (i.e. I won’t cover fixing bad sectors, but I will cover checking for bad sectors).

1) Detecting the RAID is out of sync:
cat /proc/mdstat
If the RAID is synchronized, you’ll see [UU], if not you’ll see [U_] or [_U], such as:

cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sda3[0]
      972141184 blocks super 1.2 [2/1] [U_]

md1 : active (auto-read-only) raid1 sda2[0]
      1999040 blocks super 1.2 [2/1] [U_]

md0 : active raid1 sda1[0]
      488192 blocks [2/1] [U_]

unused devices: <none>

2) Checking for bad sectors, using smartctl. NB: if you don’t have it installed (possible if your host provided you with a “minimal” install), you should be able to install it with apt-get install smartmontools on Debian/Ubuntu.
Short tests (about a few minutes each):
smartctl -d ata -t short /dev/sda
smartctl -d ata -t short /dev/sdb

Reading the results
smartctl -a /dev/sda
smartctl -a /dev/sdb

If short tests show no issues, run the longer tests (about a few hours):
smartctl -d ata -t long /dev/sda
smartctl -d ata -t long /dev/sdb

3) If there are errors on one of the drives, use hdparm to get more info to identify the drive (notably its model name and serial number), so as to be able to ask your host to change the right drive.
hdparm -I /dev/sdb
hdparm -I /dev/sda

4) To list the partitions on all disks (it can be interested to compared this to the output of cat /proc/mdstat)
fdisk -l

5) In the case I had, it turn out that my RAID had desynchronized, but no disk was damaged: only the second drive had stopped and fallen out of the RAID for some reason… So I was able to reconstruct the RAID at once, by running the following command for each partition (do not forget to adapt the names of both the RAID partition and the physical disk partition):
mdadm --manage /dev/md2 --add /dev/sdb3

And then a few snapshot of reconstruction progress (it takes quite a bit of time, of course, since all the disk is read, not just the used space):

cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sdb3[2] sda3[0]
      972141184 blocks super 1.2 [2/1] [U_]
      [=>...................]  recovery =  8.4% (82301952/972141184) finish=1017.9min speed=14568K/sec

md1 : active (auto-read-only) raid1 sda2[0]
      1999040 blocks super 1.2 [2/1] [U_]

md0 : active raid1 sda1[0]
      488192 blocks [2/1] [U_]

After the md2 partition is resynchronized:

cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sdb3[2] sda3[0]
      972141184 blocks super 1.2 [2/2] [UU]

md1 : active (auto-read-only) raid1 sda2[0]
      1999040 blocks super 1.2 [2/1] [U_]

md0 : active raid1 sda1[0]
      488192 blocks [2/1] [U_]

Adding back the md1 one:
mdadm --manage /dev/md0 --add /dev/sdb1

And now after md0 was done too:

cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sdb3[2] sda3[0]
      972141184 blocks super 1.2 [2/2] [UU]

md1 : active raid1 sdb2[2] sda2[0]
      1999040 blocks super 1.2 [2/2] [UU]

md0 : active raid1 sdb1[1] sda1[0]
      488192 blocks [2/2] [UU]

Bonus because I don’t want to lose the link and I’m not sure it deserves an entire new post: a few methods to wipe empty disk space (I usually do that before giving back the servers I rent): http://superuser.com/questions/19326/how-to-wipe-free-disk-space-in-linux

Posted in hardware, Linux, servers.


How to unban peers in qBittorrent

qBittorrent has a feature to easily block a misbehaving peer (by IP) in just 2 clicks. This is nice feature (although I don’t really use it often), but it has a big weakness: the GUI doesn’t provide any easy way to unblock the banned IPs.

Luckily, it’s quite easy to unban them manually, although I guess this may get a bit tedious if the list is long (again, it shouldn’t be though): the ban list is entirely saved in qBittorrent’s config file, which is located:
– on Windows, in %APPDATA%/qBittorrent/qBittorrent.conf
on Linux, in ~/.config/qBittorrent/qBittorrent.conf

Then find the line starting with IPFilter\BannedIPs= (that should be on or around line 106): it contains the list of banned IPs, which you can edit or empty.
Note that you should close qBittorrent while you’re doing your modifications, otherwise they’ll get overwritten.

Posted in software.

Tagged with .


How to find all visits from a company/ISP in Piwik

I don’t believe Piwik has any function to search among its statistics and logs. This is not really a big issue because, objectively, there isn’t much stuff to search for in the stats, but still sometimes I wish I could do such searches. For instance, recently I wondered how often I was visited by people from a specific company. And there’s no tool to answer that question in Piwik. However, you do have access to the full visits database. After a brief look around, the log_visit table seemed the most appropriate for my needs: its location_provider field contains the URL of the visitor’s ISP, which is often the visitor’s company itself, as long as the company is large enough to own its own IP range.

So, the query (MySQL/MariaDB/whatever) simply looks like this:
SELECT * FROM log_visit WHERE location_provider LIKE '%company.domain.name%'

Note that:
– Piwik doesn’t manage to map all IPs to companies or ISPs, you’ll probably miss like a third of them, or maybe even more.
– From the results, you’ll have easy access to information about the ID of the visited site, the referrer, and date & time of the visit, but that’s about it. You’ll have to use that information to browse the normal GUI if you want to easily read more details about any specific visit. Or do some coding if you feel inspired 😉
– For your search, if you don’t find any result, try a shorted name for the domain name, notably without the leading www, and also maybe without the final extension.

Posted in Uncategorized.

Tagged with .


How to increase video caching in VLC

I have some issues with unidentified software unreasonably trashing my hard drive on random occasions. Could be Copernic or other indexing things, yet whatever it is doesn’t show up in the task manager as a big disk I/O consumer (on a side note, Firefox does MASSIVE reads and writes: after only about 3 hours of moderate use, the task manager reports more than 10GB of reads and 1GB of writes, all this with disk caching disabled).

Anyway, those undesired disk accesses make VLC lag when playing videos. The 2 solutions I found first were to either copy the files to my SSD, or to a RAM disk. But that’s not very convenient, plus copying files to the SSD wears it down, and the RAM disk has very limited capacity since I already use it for other purposes.

Eventually, after playing a few livestreams and noticing a caching setting for them, I thought about increasing the video caching in VLC for files loaded from the disk. The only problem left was that VLC settings are a total, ever-changing mess. I found a first clue here, but the setting is no longer at the indicated location. I searched around, and around, everywhere, and finally found it. It is now directly under the Input/Codecs section, at the top of the very bottom section (Advanced). Of course, you need to switch to advanced settings (the toggle is at the bottom-left). Screenshot below:
VLC settings for file caching

As you’ll notice, the default value is 300 ms. I increased it to 5000, so 5 seconds. The only drawback of increasing it too much is that it can make starting the video (at the beginning but also on seeking) a bit slower if the bitrate is very high, but since I mainly use low bitrates it’s not an issue for me.

Update (2018-05-21)

Randomly bumping into this post almost 5 years later, I notice it mentioned my problem with excessive disk I/O and using a RAM disk as a possible solution. Well it turns out I eventually found out what was the culprit of this excessive use: it was, ironically… the RAM disk itself. It somehow configured itself to constantly back up the running image to the disk, how optimal! I’ve stopped using it a while ago, but as far as I remember I was able to configure it to not back up constantly, which solved the issue.

Update (2018-07-28)

Addendum to the previous update: my normal SSD use was about 30TB over 3 years. With the RAMdisk issue, this skyrocketed to 1TB/week (so over 5 times more). The RAMdisk writes were attributed, in the task manager, to “System”, and sadly I don’t remember how I managed to get to the exact culprit… I had a draft post titled “How to find which program is writing to disk” but it didn’t contain the solution :/

Posted in multimedia, VLC.


Hotmail/Outlook.com incoming mail servers are lying b******

Well, the title almost says it all. I’m using postfix, in a setup which I previously explained, to forward e-mails sent to myemail@mydomainnames.com to both a Hotmail and a Yahoo mailbox. I made this choice for the sake of redundancy, after getting tired of suffering from their rare yet still too frequent downtimes, but also historically because I wasn’t sure which one to use primarily. In the end, after big issues with Yahoo (mainly, block from sending e-mails by their broken anti-spam bot), I picked Hotmail.

Recently, I was expecting an e-mail, and it never arrived. Luckily, we had other ways of communicating, so I went to my now secondary e-mail box (Yahoo) and looked for it. And there it was. So what happened to the e-mail on the Hotmail/Outlook side?

1) It didn’t land in spam: I regularly check the spam folder, and although I give only a quick glimpse at most of the stuff in it, I don’t think I’d miss something in French, from someone with a French name. The spam I get is usually in English, or, less commonly, some funky language such as Russian or Chinese.

2) But it wasn’t rejected either. It was f***ing silently discarded. I eventually dug up the e-mail from my postfix log. Here’s the line:
Dec 6 10:32:46 81-7-10-146 postfix/smtp[6598]: D865A1166611: to=<[me]@hotmail.com>, orig_to=<[me]@mydomain.com>, relay=mx2.hotmail.com[65.55.92.136]:25, delay=2.8, delays=1.6/0.01/0.43/0.77, dsn=2.0.0, status=sent (250 <52A199B9.8020101@gmail.com> Queued mail for delivery)
That’s not a rejection, that’s a clear accept. Another e-mail passed just a few minutes later and it looked like this:
Dec 6 10:39:05 81-7-10-146 postfix/smtp[7457]: 65D2B1166004: to=<[me]@hotmail.com>, orig_to=<[me]@mydomain.com>, relay=mx3.hotmail.com[65.55.92.136]:25, delay=2.3, delays=1.2/0/0.44/0.6, dsn=2.0.0, status=sent (250 <52A18C48.307060@example.com> Queued mail for delivery)

So, Hotmail is silently discarding some of your e-mails, for no good reason, and not telling you about it. I dug up a bit and also found an e-mail from a merchant I recently ordered from. Not a newsletter, but a question related to the order. So definitely some other e-mail which didn’t look like spam AT ALL. What the flip does Micro$oft think they’re doing with their lame filters?

Update: just found this post of someone having the same problem, looks like I’m not the only one having trouble with them.

Posted in Microsoft, web filtering.


Various Facebook funny lines #5

Number 4 is there, below comes number 5 🙂

I have no words to describe this day. I do, however, have a ton of obscene gestures.

Remember, you’re only young once, but you can be immature forever.

I do many things well, none of which generate income.

Religion should be outlawed: just send them in for medical treatment. After all, anyone who believes in some big invisible dude who can do anything is clearly nuts.

I’m not open to many people. I’m usually quiet and I don’t really like attention. So if I like you enough to show you the real me, you must be very special.

I’m glad I don’t have to hunt for my food, I don’t even know where sandwiches live.

The government could fix everything that’s wrong with the country if they’d simply read my Facebook updates.

Sometimes, when dealing with people, you can’t help but stop and think, “Yup, I’m about to get my first assault charge”.

I danced like noone was watching, but someone was watching, thought I was having a seizure and called an ambulance.

You must be confusing me with the maid we don’t have.

Common sense is not a gift, it’s a punishment. Because you have to deal with everyone who doesn’t have it.

I’m one bad relationship away from getting 30 cats and calling it a day.

Anyone who genuinely believes it is better to be dead from murder than traumatized by rape would advocate merciful euthanasia for rape victims.

Never act without evidence: assumptions are the mother of all fuckups.

Be strong enough to stand alone, be yourself enough to stand apart, but be wise enough to stand together when the time comes.

If President Obama really wants to hurt the Syrian government, he shouldn’t send cruise missiles, he should send over some of his economic advisers.

Obama voters are the reason we have to put directions on shampoo.

Every time you make a typo, the errorists win.

Posted in funnies, Totally pointless.


Solving system error -3009 when formatting hard drive (Yast, OpenSUSE)

I got this “-3009” error when trying to install OpenSUSE on a fairly old laptop. The error occurred when formatting the partition, at the beginning of the (actual) installation procedure. I previously had a “-3008” error, which clearly stated that was because I mounted the partition (I used the liveCD and browsed around a bit before launching the installation), so I first thought this might be connected somehow. But the error even occur while running the installation procedure without loading the liveCD desktop.

Unfortunately, no indication is given apart from the error number. And searching for it with duckduckgo basically returned only this 4 year old thread, suggesting an issue related to the partition system in use (you know, MBR vs GPT or something). Anyway, in my case that wasn’t the issue. It turned out the hard drive had issues: a bunch of undetected badblocks. Luckily, the post I found and the fact that the error occurred during formatting, combined with the fact that the previous OS installed on this laptop (Ubuntu) was last seen failing to boot with a hard-drive related error directly oriented me to this. However, there remained the question of how to fix the drive.

I found those 3 posts dealing with the subject:
The Beginner’s Guide to Linux Disk Utilities
Forcing a hard disk to reallocate bad sectors
How to securely keep a hard drive with bad blocks in a raid array

They present a lot of different approaches, not always very clearly explained, and notably not clearly stating the differences between the approaches. Feel free to browse them if you need to try more stuff. Personally, I first tried a combination of badblocks and fsck: badblocks to generate a list of faulty sectors, and fsck to mark them as faulty. I’m not sure it really worked, because some of the blocks detected by badblocks were reported as out of range by fsck. Nonetheless those are the commands I used to do it, as far as I remember (I think the second one needed some tweaking, but I don’t remember it :/):
sudo badblocks /dev/sda5 > /path/to/list_of_bad_blocks
sudo fsck.ext4 -C 0 -l /path/to/list_of_bad_blocks -y /dev/sda4

NB: before you do this, make sure the partition you’re checking is unmounted: ummout /dev/sda4
NB2: don’t forget to replace file names, drive names and partition number by yours
NB3: the partition of interest was previously formatted as ext4. I think it should work on ext2 and ext3, but I’m not sure about other types such as NTFS.

Since this gave me a few error messages about out of range sectors, I assumed it missed and directly decided to try another method. Fsck is supposed to be able to both check and repair, only somehow the man page and –help aren’t very useful about it. Luckily, when trying one of the commands mentioned in the above-linked posts, I eventually reached a more helpful help page. The argument to check a partition with fsck is simply “-c”. Giving something like:
sudo fsck -c -t ext4 /dev/sda4
NB: again, I did some more tweaking (notably, I believe there’s an option to automatically fix errors without user input), but I forgot it :/

Well, that’s it. After that the partition was good enough to be formatted and have OpenSUSE 13.1 installed without issues. Sorry for the lack of precision in the commands, that was like a week ago and I didn’t expect to recall those details so poorly after so little time… (and the commands can’t be found in my history since I used them all from the LiveCD) Anyway, I still hope you’ll find them helpful 🙂

Posted in hardware, Linux.


Configuring a self-restoring reverse SSH tunnel

No time to make it a proper post, so I’ll mainly post the syntax and files to edit. Here I’m creating a reverse SSH tunnel to be able to access my GNU/Linux machine behind a work firewall, via another Linux machine (a VPS or dedicated server running Linux too) as a relay. Then I use any PC (it’s really as easy on Windows and Linux for this part) with PuTTY to connect to it. If the machine behind the firewall is on Windows and you know Windows enough to modify the batch file accordingly and add it at start-up, you’ll probably still find the guide useful. If the machine you want to use as a relay runs on Windows, well good luck setting up OpenSSH on it… 🙁

List of stuff to install (if not already present):
On the relay: openssh-server, webmin (optional)
On the firewalled machine: plink (in package putty-tools I think), sudo

On the relay, make sure OpenSSH allows opening gateway ports. The config file is /etc/ssh/sshd_config. Just add or modify:
GatewayPorts yes
NB: in previous guides, I used to set this to GatewayPorts clientspecified, but for some reason (maybe I lost a plink option or something), at the moment “clientspecified” is not working.

Optionally, still on the relay, create a user just for the tunneling (useradd XX, then password XX). This way you can give it a specific password (for the sake of simplicity I won’t be using a key but a password) and disable the shell, so as to only allow tunneling but note typing commands. To do so, we’ll use a shell I found here and there: create for instance a file named /usr/bin/tunnel_shell with the following content:

#!/bin/bash
clear
echo -e "\r\n\033[32mSSH tunnel started, shell disabled by the system administrator\r\n"
while [ true ] ; do
sleep 1000
done
exit 0

Then make sure it’s executable (for instance chmod 755), and set it as the shell for the tunnel-specific user (for this I used Webmin, in System → Users and Groups).

On the firewalled machine, create a bash script (eg /home/wrapper.sh) containing something like:

#!/bin/bash
while [ true ] ; do
plink -ssh -P 22 -pw [password] -N -R 4444:127.0.0.1:22 [usernam]@[relayIP]
sleep 120
done
exit 0

It’s an infinite loop, that tries to connect every 120 seconds to [relayIP] (IP of the relay machine). 4444 is the port on [relayIP] that will be forwarded to port 22 on the firewalled machine (you can choose any port you like, of course). Check out the plink documentation if some arguments are unclear.
You could now test the command alone (the whole “plink” line) to see if it properly connects to your relay and establishes the tunnel (try connecting to [relayIP]:22).

Finally, we will add the script to be launched at start-up on the firewalled machine. First I advise you to create a specific user that will launch the script, so that it will be easier to spot in top (there’s no easy way to stop the script, so the simplest way I found is to get its PIDs via top and kill them).
Then in Webmin, go to System → Bootup and Shutdown, then Create a new bootup and shutdown action. Enter a short name (eg my_tunnel.sh), a description (eg My reverse tunnel), and in the Bootup commands field, enter sudo -u [username] /home/wrapper.sh. Of course, enable Start at boot time.
NB: when you validate, Webmin will create a little wrapper script around your command. But as of version 1.660, if you leave the stop block empty, it will neglect to add a
"# Default-Stop:" line in the INIT INFO section. You should add it manually if you want to avoid this kind of warning:
insserv: missing `Default-Stop:' entry: please add even if empty.
insserv: Script wrapper.sh is broken: incomplete LSB comment.

That’s about it. Next time you restart the firewalled machine, it will connect to the relay and establish the reverse tunnel, and you can connect with PuTTY to [relayIP]:4444. Note that the first time you connect, plink will ask you to confirm the relay’s RSA key: this requires manual input (you need to respond yes), so before running the script you should establish a connection manually, with the same user as the one which will be running the tunnel automatically. Long story short, just run sudo -u [username] /home/wrapper.sh yourself before letting the script do it itself.

Posted in Linux, web filtering.

Tagged with .