12.15.2011

Flush Jaspersoft OLAP Cache from Talend/JasperETL

JaspersoftETL is a Java code generation tool designed to load your operational data into a data warehouse and get it ready for reporting and analysis.  Jaspersoft OLAP takes advantage of aggregate tables and an in-memory cache to speed up queries. If you're running a nightly job to load the DW, it's a good idea to clear the OLAP cache at that time, otherwise your users will not take advantage of new data.

There are several ways to clear the OLAP cache:
  • Login with superuser privileges and display the OLAP Settings page.
  • In the OLAP Settings panel, click Flush OLAP Cache
  •  Using utilities such as cURL and wget, send an HTTP request to this URL: http://server:8080/olap/flush.html?j_username=superuser&j_password=
  • Restart the application server instance. 
  • Call the Java API.
I've created a small component that allows the clearing of cache via the URL method from a JasperETL/Talend job. Essentially it will prompt you for a username and password (and note that only superuser can clear the cache by default) and a server URL. The URL should point to something like http://server:8080/jasperserver-pro/olap/flush.html

You may find the tJasperOLAPCache component on talendforge.org

Another useful thing to do would be to use the tJasperServerExec plugin to re-populate the cache.

12.01.2011

Using tGoogleAnalyticsInput in JasperETL/Talend

This is a short article on getting Google Analytics into JaspersoftETL/Talend ETL tools. Google Analytics has a nice interface and many great ways of analyzing web traffic, it'd be hard to beat the work Google did there. The purpose of this connector would be to bring the data into your own data warehouse and join it with other intelligence you might have. A great way to display the aggregated data would be with a BI tool like Jaspersoft.

I'm writing because the help for the tGoogleAnalyticsInput is in French and incomplete.

So let's get started. You'll need JasperETL 4.1.2 or older (commercial editions here, open source here).

Installing the plugin:
Create a new Job Design and drag the tGoogleAnalyticsInput component from the Palette (under Internet):


















Configure the component. First start with finding some fields you'd like to use from the Google API: http://code.google.com/apis/analytics/docs/gdata/dimsmets/dimsmets.html. Note that you need to have some metrics (can't be all dimensions, vice versa).

Here's an example schema (you'll have to put in the fields yourself):
Now configure the component. In order to get your Profile ID (note it doesn't start with UA) read the cryptic Google Documentation about it. I got mine by examining the URL while logged into GA and looking at the &id= parameter.

The "value" column comes directly from the Google API and will also have to be filled in. I've also found that I HAVE to filter on period. You can do 1980-2020 if you want to include everything. 

Now you can run the the job and see your visitors:

Funny enough. If you're reading this right now then you will eventually flow through the GA database and the very job I'm displaying above ;)

8.01.2011

Icinga Reporting with JasperReports Server - Part 2: Usage + Basic Report Development

Short tutorial, shows how to schedule a report, how to change the way you are prompted for parameters and some basic editing of reports within iReport

Icinga Reporting with JasperReports Server - Part 1: Installation

A quick overview of how to install and configure the reporting add-on packages for Icinga (a Nagios Monitoring Fork), written for JasperReports Server.


4.18.2011

Backing Up JasperServer in Linux

This is a sysadmin how-to for backing up JasperReports Server (formerly JasperServer) and it's database (MySQL only for now).

I wrote a fairly robust script to achieve this task. It will read the database configuration from your default_master.properties file, work with several versions of JasperServer, delete old backups, write to the syslog and export the repository contents to file.

First drop my backup script into your \buildomatic folder if you're using 4.0 and abovie, 3.7 and below put in your /scripts folder.

Once the script is in place edit the variables at the top of the script:

This is to signify that you're using JasperServer 3.7.1 and below:
IS_PRE_4=false

This is a name for the product:
PRODUCT_NAME="JRS-4.0-Pro"

This is where backups will be stored:
BACKUP_DEST="/home/ernesto/Dropbox/Backups"

This will delete old backups if set to true
DELETE_OLD=true

How many days to keep old backups
DAYS=120

Now add it to your crontab. I will do it once a week in a lazy manner so will add the script to vim /etc/cron.weekly/js-backups

#!/bin/bash
/usr/local/jaspersoft/jasperserver4/buildomatic/js-backup.sh


The script will log to /var/log/messages:


The output files will look like this, I chose to stick them in my Dropbox folder for off-site backups:










You can download the script from this jasperforge forum post or view it below:

4.01.2011

Music Player Daemon w/ lame Support in Debian

Trying to get my old Blackberry to stream from my mpd server, it turns out the built in http stream output is not compiled with lame encoder (mp3) support. Here are some instructions on building the source package with the lame encoder options compiled in for Debian Squeeze (6.0)

(as root)
Install libmp3lame0 and lame from http://debian-multimedia.org/
cd /tmp/
Get the build dependencies for mpd:
apt-get build-dep mpd
Get the mpd source package:
apt-get source mpd 
cd mpd-0.15.12 (your version may differ)
Edit the file debian/rules and remove --disable-lame
Build the binary:
./debian/rules binary
Build the .deb package:

cd .. 
dpkg-source -b mpd-0.15.12
Install the package:

dpkg -i mpd_0.15.12-1.1_amd64.deb
Lock the package in place so that updates won't blow away your change:
echo mpd hold | dpkg --set-selections
That's it! Then you can configure an audio output in /etc/mpd.conf:
audio_output {
        type            "httpd"
        name            "My HTTP Stream"
        encoder         "lame"
        port            "8000"
        quality         "5.0"     
        always_on       "yes"  #This option doesn't seem to work for me :(
}

3.24.2011

ThinkPad Overheating Issue Fix

She's pretty hot!
On my ThinkPad T410s fans were disabled by default after installing Debian 6 (Squeeze).

I would randomly be working on CPU intensive stuff and the computer would shut down with:
Critical temperature reached (128 C)
 To fix this I created the file /etc/modprobe.d/thinkpad_acpi.conf and added options thinkpad_acpi fan_control=1 to it. I've submitted a bug against the acpi package in Debian.

I got help from ThinkWiki on this.




3.09.2011

File Management Tip for Nautilus/Gnome in Debian

nautilus-open-terminal in action
Often I find myself browsing to a directory in Nautilus and then realizing that I'd rather be in terminal. Other times I want to be in Nautilus when I'm in the terminal. There's certain things the command line is better at than the file browser and vice versa. Switching between the two is fairly easy in Gnome/Nautilus.

Going from Nautilus to Terminal:
You'll need the nautilus-open-terminal package ( sudo apt-get install nautilus-open-terminal; killall -HUP nautilus ). Then in any Nautilus, right click on white space and select "Open in Terminal"

Going from Terminal to Nautilus:
This one is simpler. I watched a guy at work do this on his Mac and got jealous. You can go from Terminal to Nautilus opened in your current directory by typing: nautilus .


ernesto@debian-t410s:~$ nautilus .

1.16.2011

Linux zd1211 Wireless Speed Fix

For some reason my Belkin 54Mbps USB wifi adapter operates at 1 Mb/s under Linux.
It's detected as the following by lsusb:

Bus 001 Device 002: ID 050d:705c Belkin Components F5D7050 Wireless G Adapter v4000 [Zydas ZD1211B]

If I do a iwconfig wlan0 rate 54M the card starts working at 54Mb/s. To fix this more permanently I added the following to my /etc/network/interfaces file:

auto wlan0
iface wlan0 inet dhcp
        wpa-ssid 56k
        wpa-psk secret
        wireless-rate 54M