12.07.2010

Solaris on Virtualbox

Trying out Solaris on Virtualbox to troubleshoot an issue at work with no luck booting. The VM would hang after the grub prompt.

Disabling USB support and sound support did it for me.

11.05.2010

Cron Jobs in the Green Data Center

Yesterday I wrote a script to monitor a UPS (running apcupdsd) server using Nagios. Today I'm looking at some of the results of the performance data. It seems that the hourly cron jobs on my server actually cause some power usage. I'll have to think over how often it's actually necessary to run those scripts given it's sucking a noticable bit of juice. Check out the graph (produced by PNP for Nagios)

4 hour Load Percentage of our new APC 2200 @ work

10.07.2010

SSH Client Keep Alives

At my new office in the Shell Building the main router is a SonicWall which has a setting to drop idle TCP connections after 15 minutes. This is annoying since I usually have several going at once. I don't have access to the router so I had to work around the issue.

The poor man's solution is to run the "top" command on your idle terminals so that packets come through. Not elegant and too manual for me.

The rich man's solution is to change the server's keep alive settings (TCPKeepAlive yes). I call this the rich man's solution because it means every one bends over for you.

The middle class man's solution is to add some directives to the default SSH client options on your local machine. Here's how:

Edit ( as root) /etc/ssh/ssh_config and add the following lines:

TCPKeepAlive yes
ServerAliveInterval  60
ServerAliveCountMax 5


This means that every 60 seconds your machine will send a keep alive to the server if there is no other activity. If the physical connection gets dropped then every 60 seconds the keep alive will be sent until it's done that 5 times, finally giving up (in 5 minutes).

From the man page:

TCPKeepAlive
Specifies whether the system should send TCP keepalive messages to the other side. If they are sent, death of the connection or crash of one of the machines will be properly noticed. However, this means that connections will die if the route is down temporarily, and some people find it annoying. The default is ``yes'' (to send TCP keepalive messages), and the client will notice if the network goes down or the remote host dies. This is important in scripts, and many users want it too. To disable TCP keepalive messages, the value should be set to ``no''.
ServerAliveCountMax
Sets the number of server alive messages (see below) which may be sent without ssh(1) receiving any messages back from the server. If this threshold is reached while server alive messages are being sent, ssh will disconnect from the server, terminating the session. It is important to note that the use of server alive messages is very different from TCPKeepAlive (below). The server alive messages are sent through the encrypted channel and therefore will not be spoofable. The TCP keepalive option enabled by TCPKeepAlive is spoofable. The server alive mechanism is valuable when the client or server depend on knowing when a connection has become inactive. The default value is 3. If, for example, ServerAliveInterval (see below) is set to 15 and ServerAliveCountMax is left at the default, if the server becomes unresponsive, ssh will disconnect after approximately 45 seconds. This option applies to protocol version 2 only.
ServerAliveInterval
Sets a timeout interval in seconds after which if no data has been received from the server, ssh(1) will send a message through the encrypted channel to request a response from the server. The default is 0, indicating that these messages will not be sent to the server. This option applies to protocol version 2 only.

9.17.2010

Adobe Flash is back for amd64!

Finally the 64 bit edition of Flash is somewhat available again! It's called Flash "Square". I tried it last night on my dual core Atom HTPC running Debian Squeeze. It was amazing. Full screen HD YouTube videos play smooth as butter. I've never seen this with Flash in Linux! Amazing!

By the way I still hate Flash but you gotta live with it so...

See my edits in the Debian wiki on how to install on a 64 bit platform:
http://wiki.debian.org/FlashPlayer#Debian5.0.27Lenny.27and.27Squeeze.27amd64

d.o.t.s in your Gmail address demystified

This is probably not a surpsrise to anyone but me but I still want to share.

According to Google:
    • homerjsimpson@gmail.com = hom.er.j.sim.ps.on@gmail.com
    • homerjsimpson@gmail.com = HOMERJSIMPSON@gmail.com
    • homerjsimpson@gmail.com = Homer.J.Simpson@gmail.com
    Sweet! Now to make it even harder to contact me just e-mail me at e(dot)r(dot)n(dot)e(dot)s(dot)t(dot)o(dot)o(dot)n(dot)g(dot)a(dot)r(dot)o AT gmail(dot)com

    8.30.2010

    1,741 times faster than dial up!

    I am checking out the grand new offices for the RAMCloud project at Stanford. The internet here is 1,741 times faster than 56k dial-up! Ironically storage speeds have not grown at the same rate as Internet speeds, that's what RAMCloud hopes to solve (in the data center).

    This is insane bandwidth. Oh and every PC here gets a public IP! I'm in Internet heaven.

    8.17.2010

    Open Government Data and JasperReports

    Major geek out this evening. I was supposed to go to the gym with Diego but he passed out on the couch from jet lag and I decided to hit the open source reporting space instead (the obvious choice!). I'm evaluating JasperReports, an open source reporting engine. I'm very impressed with the results.

    Installation:
    Installation was a breeze - I fired up a Debian 5 VPS on my home server and after downloading the JasperServer package I had my TomCat, MySQL, Java installs all done for me. I had the option to configure each component separately - I opted to have Jasper do all that for me this time. I also decided to give JasperReports-Pro a try under the time trial demo. There's also the completely free open-source edition.

    Data Source:
    Once I was able to log into the web interface I had to find some data! I decided to try data.gov. I found some "raw data" on 2008 adoptions per foreign country for United States families. The data came in the form of an .xls file that I cleaned up, turned into a .csv and imported into a separate MySQL server that I had running.

    Configuring JasperReports:
    Configuring JasperReports was not too bad. There's definitely a learning curve but within a couple of hours I was able to get everything plugged in nicely and a report generated. The basic steps were:

    1) Create a JDBC connection to your database (in this case it was another host on my LAN with a mysql server listening for connections). Jasper calls this a JBDC Data Source

    JasperReports: Turning Data into Information
    2) Create an abstraction of the Data Source and the queries. This can get extremely complicated - you can pull in several tables and several data sources, manipulate them in any way you like (yes, graphically if you like) and then you should have what Jasper calls a Domain. My Domain was very simple, just pulls in the tables from my data source. In the real world this step would be done by an experienced database administrator.

    3) Finally you can then set up a report using the data from the Domain created in step two. The idea is that your end users create reports using the JasperServer GUI, the end user really doesn't need to worry about the JBDC data source.

    The Results:
    After all this work you have a report. A sweet report that is linked to your data source. If I upload data into my database the reports are updated instantly. This is very cool. Since I already have all the data from my table in my Domain, creating other reports and mash-ups is simple.


    The "real-world" application doesn't seem obvious in my example here - if you work in an enterprise where data is needed on a constant basis then JasperReports is for you. Reports can be ran on demand by end-users or scheduled and e-mailed on a recurring basis. Here's what my report ended up looking like. It's super basic and just tells me that Guatemalans were the child of choice for 2008.

    If you put together all the information available from the government on sites like data.gov and combine it with the power of a tool like JasperReports, having incredibly useful information at your fingertips becomes a reality. 

    7.21.2010

    Dropbox in Debian Squeeze

    I decided to give Debian Lenny +1 distribution, Squeeze a try on my work desktop. Things look really good and easy for one of my favorite cloud services, Dropbox. To install (instead of the old manual methods) just enable the non-free repository in your /etc/apt/sources.list:
    deb http://mirrors.kernel.org/debian/ squeeze main contrib non-free
    Then you can simply install Dropbox with:
    apt-get install dropbox

    7.12.2010

    Finally a fix for home, end, page up and page down keys in Mac OS Terminal!

    Thanks Mac Improved Blog man:

    http://macimproved.wordpress.com/2010/01/04/fix-page-updown-home-end-in-terminal/

    7.09.2010

    lighttpd: whitelist some IPs while authenticating the rest

    Here's the scenario: you have an office full of people that need access to a certain web app. Some of them probably have insecure passwords and you're too busy to worry about the latest security holes in your web-app. Slow down attackers by allowing your office IP addresses in while denying the open web access until they put in a simple group password.

    In other words, this post walks you through having lighttpd allow some IP addresses in (and authenticating with your web app) and others to have to authenticate with mod_auth first, then the web app.

    These instructions were tested on Debian Lenny:
    • First Enable the authentication module:
      lighttpd-enable-mod auth
    • Create the password file, the format is username:password
      vim /etc/lighttpd.user
      Make the password file owned by the webserver user:
      chown www-data:www-data /etc/lighttpd.user
    • Configure the auth module:
      vim /etc/lighttpd/conf-enabled/05-auth.conf
      * Comment out the auth.backend = "plain" line
      * Comment out the auth.backend.plain.userfile = .... line
      * Change the auth.backend.plain.userfile file to the one you created above, /etc/lighttpd.user
    • Finally, have all IPs authenticated, except for the IP1 and IP2 (add more separated by pipes) by adding the following to /etc/lighttpd/lighttpd.conf:

      $HTTP["remoteip"] !~ "IP1|IP2" {
      auth.require = ( "" =>
      (
      "method" => "basic",
      "realm" => "Employees Only!",
      "require" => "user=username"
      )
      )
      }

      Note: you can't use hostnames, only IPs
    • Reload lighttpd and you're done!


    5.10.2010

    Pico Projectors

    Is the next big (little) thing Pico Projectors? Check out my friend's site http://picoprojector.org/ - he's got pretty comprehensive reviews and news about the miniaturized projectors.

    3.23.2010

    Blackberry Bold 9700 and Mac OS X 10.6.2 Bluetooth Tether for AT&T

    This is a fairly lazy post. I'm assuming you have the devices tethered via Bluetooth - that's fairly common knowledge. What's uncommon is getting it to work. I tried the Blackberry Modem profiles that people cooked up with no luck. What ended up working was this Option N.V device profile. Just follow the screenshots.


    * Note password is CINGULAR1


    The speeds are under a Mbit up and down. Not great but this works. If this isn't working I suggest you tail -f /var/log/system.log while "Use verbose logging" is checked in the PPP properties.

    Source here (obscure CrackBerry post)

    2.25.2010

    Troubleshooting Cisco/Linksys SPA-942 phones with a syslog Server

    Chances are you're using an Asterisk server on Linux with your SPA-941, SPA-942, or SPA-962 VoIP phones. If you're having an issue, like trouble upgrading firmware it's nice to know what the phone is saying. Simple.

    Enable remote syslog logging:
    On your Linux machine, in this case CentOS, enable remote syslog logging in /etc/sysconfig/syslog by adding a -r to the SYSLOGD_OPTIONS line. It will end up looking something like this:
    SYSLOGD_OPTIONS="-r -m 0"

    Restart syslog (/etc/init.d/syslog restart) and then onto the phone.

    Enable Logging on the SPA device
    On your SPA device under (Admin -> System) enter the IP of your syslog server into the boxes and set a debug level (0 is off, 3 is most verbose):


    Look at your logs:
    In /var/log/messages you should see some output the phone is putting out. This is an example of a successful firmware upgrade on an SPA-941
    Feb 25 13:06:18 10.0.22.114 SPA-941 00:0e:08:23:15:2c -- Requesting upgrade http://10.0.22.15:80/spa941-5-1-8.bin
    Feb 25 13:06:19 10.0.22.114 fprv_upgrade_from_buffer_priv(int manual:0, unsigned char *buf: kOsMoS9 fIrMwArE, int file_len: 757258)
    Feb 25 13:06:19 10.0.22.114 fprv_upgrade_from_buffer_priv(int manual:0, unsigned char *buf: kOsMoS9 fIrMwArE, int file_len: 757258)
    Feb 25 13:06:19 10.0.22.114 ** old magic: kOsMoS9 fIrMwArE, new magic: kOsMoS9 fIrMwArE
    Feb 25 13:06:19 10.0.22.114 ** old magic: kOsMoS9 fIrMwArE, new magic: kOsMoS9 fIrMwArE
    Feb 25 13:06:44 10.0.22.114 SPA-941 00:0e:08:23:15:2c -- Successful upgrade http://10.0.22.15:80/spa941-5-1-8.bin -- new image 5.1.8

    There's not a lot of documentation from Cisco/Linksys on error codes and reboot reasons..on your own there!

    2.16.2010

    Batch convert Asterisk GSM WAV files to mp3


    I am working on a fun project at work to provide web based visual voice-mail for a ticketing system. I realized Flash audio players are not able to play WAV files so had to work around that. Since users are also using voicemail files in other ways I can't just change the output format from Asterisk. The type of file that I'm working with is identifed with file ms0012.WAV as: msg0012.WAV: RIFF (little-endian) data, WAVE audio, GSM 6.10, mono 8000 Hz


    First I'll say that I found this post http://www.thiscoolsite.com/?p=73 but like some of the commenters I couldn't get the script to work. The author assumes some other format than Asterisk spits out by default. Lame would complain that Unsupported data format: 0x0031



    The Tools
    Sox: http://sox.sourceforge.net/
    Lame: http://lame.sourceforge.net/

    So here's what I do:
    Convert GSM encoded wav to Microsoft PCM
    sox msg0012.WAV -s msg0012.wav

    Convert the PCM wav to mp3:
    lame msg0012.wav msg0012.mp3

    And here's a script to do this as a batch:

    2.02.2010

    Delete multiple files in Subversion (Linux)

    Removed a ton of files from an SVN repo? This will delete all the files that show up with a ! when you do svn status.

    svn rm $( svn status | sed -e '/^!/!d' -e 's/^!//' )

    Thanks to Snippler for the snippet.

    1.25.2010

    Official Google Reader Blog: Follow changes to any website

    Official Google Reader Blog: Follow changes to any website

    1.18.2010

    Add sbin to user PATH in Debian

    Debian does not add sbin to your path by default. I have no idea why. I know sbin is supposed to be administrative tools that you would only want to run as root, however, there are some useful ones that don't require root, and even if they need root, why hide them from useful tools like auto complete?

    Apparentely it's not a new disucssion - this Debian mailing list thread dates back to a decade ago!

    Regardless, I want it in my path - the easiest way that I've found is to modify the top of /etc/profile to remove the if statement that sets path if you're user id 0 (root) or not:

    Before:

    if [ "`id -u`" -eq 0 ]; then
    PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    else
    PATH="/usr/local/bin:/usr/bin:/bin:/usr/games"
    fi


    After:

    PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"


    Save the file, log out and log back in and you're all set. Note that this affects all users on your system.

    1.04.2010

    Modifying Headers for testing Virtual Hosting

    While working on a website hosted on lighty's virtual hosting environment I realized I wanted a natural way to test the site before pointing the DNS over. If you're sharing one IP for multiple sites then this tip is useful. It uses a Firefox add-on.

    Get the "Modify Headers" Firefox add-on here.

    Once installed go to Tools -> Modify Headers


    • Action -> Modify

    • Name -> Host

    • Value -> Domain you want to "spoof"



    See a list of other headers you can play with here.

    1.03.2010

    Type accents and other Latin American punctuation on your Mac

    I'm helping type some of my grandfather's writing from the 1960s (written on a typewriter, not even worth trying OCR) and I needed some punctuation love on my MacBook, this is what I found.

    Accents:
    To write accents on letters, just tap Option (alt) + e and then the letter:


















    áoption + ethen a
    éoption + ethen e
    íoption + ethen i
    óoption + ethen o
    úoption + ethen u




    Enyay (ñ)
    For ñ, tap Option (alt) + n and then n.

    Punctuation:









    ¡option + 1
    ¿option + shift + ?






    Read more on Apple's help page.