Irrigation 2.0 with Rain8Net

The problem: Automatic sprinklers are great for convenience but waste a ton of water. You see businesses and neighbors running sprinklers during a rainstorm all the time–right? Rain conscious individuals can turn their sprinklers off when not needed, but you lose convenience and sometimes it is a pain to get back on track.

The solution: Rain8Net from WGL Designs, a PC with a serial port, and a little bit of Ruby programming.

Disclaimer: this post is a bit premature. I just got my Rain8Net last weekend and spent some time trying to program it. I plan to script the whole system so that it reads weather reports from the Internet to determine irrigation needs. That part isn’t ready yet. This is just an introduction to what I’ve discovered so far.

That being said, here is a sample of how to communicate with the Rain8Net via Ruby. (First, be sure to download and install the ruby-serialport library.)

require 'serialport.so'
tty = 0
rain8 = SerialPort.new(tty, 4800, 8, 1, SerialPort::NONE)
# Turn on Zone 1
rain8.write(["400131"].pack("H*"))
sleep(60)
# Turn off Zone 1
rain8.write(["400141"].pack("H*"))

WLG Designs has great documentation explaining what the various codes do. I am providing the code above as an example of how to implement the provided codes. (It took me many tries to get this far. Hopefully it will save someone else a bit of work.)

I plan to develop a Ruby library for use with the Rain8Net which will make it much easier to use. Watch for it…

RFC_READ_TABLE with Ruby and SAP::Rfc

Warning: another code sample is included in this post.

I spent a few hours this morning trying to use Piers Harding’s SAP/Rfc library for Ruby to read a table from SAP. I found several examples using other languages (Perl, VBscript, PHP, etc.), but the only Ruby example I could find reads the entire table. Figuring out how to load the “options” took some trial and error.

The idea of this example is to read the “LQUA” table in SAP which stores information about where to find a particular material in the warehouse. The whole thing is wrapped up in its own model so it can be easily called elsewhere in my Rails app:

class SapMaterial < SAP4Rails::Base
  function_module :RFC_READ_TABLE
  class << self

    def find_stock(options={})
      material = options[:material]
      return nil if material.blank?
      rfc = self.RFC_READ_TABLE
      rfc.query_table.value = "LQUA"
      rfc.delimiter.value = "|"
      rfc.options.value = ["MATNR EQ '#{material}'"]
      # optional set of fields to return from the table
      #rfc.fields.value = ['MATNR', ...]
      rfc.call()
      rfc
    end

  end
end

This is called with something like this:

stock_locations = SapMaterial.find_stock(:material=>'VOC300V')

Which yields a handy data set containing all the locations and available quantity for the DigiTech Vocalist 300 in the warehouse. This will be used as part of my new scan gun application which directs the shipping department to the various storage bins for picking large orders.

UPDATE: It seems the preferred approach is to use the new SAP Netweaver RFC library (sapnwrfc.rb). This changes the code sample a bit…

class SapMaterial < SAP4Rails::NW::Base
  function_module :RFC_READ_TABLE
  class << self

    def find_stock(options={})
      material = options[:material]
      return nil if material.blank?
      rfc = self.RFC_READ_TABLE.new_function_call
      rfc.QUERY_TABLE = "LQUA"
      rfc.DELIMITER = "|"
      rfc.OPTIONS = [{'TEXT' => "MATNR EQ '#{material}'"}]
      # optional set of fields to return from the table
      #rfc.FIELDS = [{'FIELDNAME' => 'MATNR'}, {'FIELDNAME' => ...}]
      rfc.invoke
      rfc.DATA
    end

  end
end

The trickiest part was figuring out the correct syntax for the OPTIONS and FIELDS. Pier’s documentation hints at the correct format (look under the heading ‘A Closer Look At Handling Parameters’), but I had to dig in to the table structures within the function in SAP to figure out that it was expecting ‘TEXT’ and ‘FIELDNAME’. So, if you’re using some other function, then explore the table structures to learn what the function expects to see.

Hostmonster on Rails

A client of mine recently moved their web hosting to Hostmonster, and I was tasked with migrating a Ruby on Rails application to the new server. I ran into a few snags.

First of all, I followed their tutorial.

The default rails welcome page worked fine, but I kept getting “Application Error” on everything else. The logs showed nothing. I found several message board posts pointing to the permissions on the “public” folder.  Dr. Chuck had the best article on the subject.

Dr. Chuck’s instructions fixed the problem for a month or two.  Then it stopped working again. I figured some server-side process had changed the permissions on my files. So, I set it up again. It still didn’t work.

I actually had to contact tech support. A couple of days later they responded with the solution (thankfully, this isn’t a mission-critical app). Hostmonster had recently updated to a newer version of Rails. So, I just commented out the version in the rails environment file:

#config/environment.rb
#...
#RAILS_GEM_VERSION = '1.2.5' unless defined? RAILS_GEM_VERSION

I believe that tells the application to use whatever version of rails it can find (the most recent version on the server). Since version 1.2.5 had apparently been replaced with 1.2.6, removing the declaration for 1.2.5 fixed the problem.

The part that still confuses me is, I had tried generating the application from scratch while logged into the Hostmonster server (via ssh). Doing so still generated an environment.rb file for version 1.2.5.

Of course, the other part that confuses me is why did 1.2.5 stop working?  When I update my own servers with ‘gem update’, I keep old versions available for backwards compatibility of existing rails apps. As far as I know, there is no problem doing it this way. Am I wrong?

Converting DivX files to mpegs for an NTSC DVD

Problem:

You have a bunch of DivX avi files and want to put them on a DVD for a friend. The friend wants to play the final product on a DVD player (not on a PC).

Solution:

Believe it or not, my first inclination was to fire up my old Windows XP computer because it has all of the Sony programs on it–including DVD Architect. To make a long story short, Windows proved itself once again as an inferior system. The DivX files have to be converted to mpeg. I tried several converters which ran for hours at a time. I finally had 3 converted files ready to burn. I thought everything worked fine until I played the DVD on a machine. It looked horrible and skipped a lot…unacceptable.

Fortunately I have Linux which helped produce a much better final product. My originals were all 16:9 widescreen, but I wanted to make them work on a 4:3 system. After installing ffmpeg (on Fedora, do this: sudo yum install ffmpeg -y), here is the command I used to do the conversion:

ffmpeg -i originalfile.avi -target ntsc-dvd -aspect 4:3 -s 720x400 -padtop 40 -padbottom 40 outputfile.mpg

Looks hairy, but it works well. I discovered the actual size of my original files are 720×400 by looking at the output of ffmpeg. So I kept this size and padded 40 to the top and bottom to create the letterbox. For reference, here some sample output from ffmpeg:


Seems stream 0 codec frame rate differs from container frame rate: 30000.00 (30000/1) -> 23.98 (24000/1001)
Input #0, avi, from 'filename.avi':
Duration: 00:44:12.1, start: 0.000000, bitrate: 1239 kb/s
Stream #0.0: Video: mpeg4, yuv420p, 720x400, 23.98 fps(r)
Stream #0.1: Audio: mp3, 44100 Hz, stereo, 112 kb/s
Output #0, dvd, to 'filename.mpg':
Stream #0.0: Video: mpeg2video, yuv420p, 720x480, q=2-31, 6000 kb/s, 29.97 fps(c)
Stream #0.1: Audio: ac3, 48000 Hz, stereo, 448 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #0.1 -> #0.1
Press [q] to stop encoding

Then I decided to wrap it in a little perl script so I could do all 24 of the files without sitting around watching the progress:


$dir = '/home/adam/';
chdir($dir);
opendir(DIR, $dir);
@files = readdir(DIR);
closedir DIR;


foreach (@files) {
next unless $_ =~ m/^([\w\.]*)\.avi$/i;
# This should all be on one line...
print `ffmpeg -i $_ -target ntsc-dvd -aspect 4:3 -s 720x400 -padtop 40 -padbottom 40 -y $1.mpg`;
}

As usual, this is mostly for my reference, but maybe somebody else will also find it useful.

Scripting Access to America First Credit Union

Here’s a little script I thought might be useful for others out there. I run it each morning using a cron job. It logs in to my account at America First Credit Union and sends me a summary of recent account activity. (You’ll need to change some of the variables if you decide to use it.)

NOTE: storing your account number, PIN, etc. in plain text is a bad idea. Don’t do it. That being said, here’s the script where you might want to…

#!/usr/bin/perl

# Some modules to help things get started (available from http://search.cpan.org)
use Date::Calc qw(Today);
use Mail::Sender;
use WWW::Mechanize;
use Data::Dumper;

##################################################################
# User settings...
#
# Enter your account number here:
$user_id = '12345678';
# Enter your PIN here:
$user_pin = '1234';
# Replace with your security questions & answers: several questions could have been
# selected during the registration process. You'll need to edit the regular expressions
# and the corresponding answers so they match those from your account registration.
$secure_qa = {'favorite TV show' => 'Show Title',
'favorite movie' => 'Movie Title',
'favorite activity' => 'Activity Name'};
# One more piece of security...your passphrase which you supplied when registering.
# The browser will show this to you so you know you're contacting the correct site.
$pass_phrase = 'YOUR PHRASE HERE';
# Email configuration. Enter what works for your environment...
$smtp_server = 'mail.myhost.com';
$smtp_user = 'username';
$smtp_pwd = 'password';
$email_from = 'me@myself.com';
$email_to = 'me@myself.com';
$email_cc_list = 'wife@something.com',
# End of user settings
##################################################################


$email_summary = '';

# Setup mech object
$browser = new WWW::Mechanize;
$browser->agent_alias('Linux Mozilla');

# Load the AFCU login screen, enter the account # and click the button...
$browser->get("https://webaccess.americafirst.com/jsp/login.jsp");
$browser->form_number(2);
$browser->field("user_id", $user_id);
$browser->click_button(name => 'login_button');

# The login procedure bounces around a few pages before getting to the PIN...
$content = 'transactiontype=loginBean&USERID='.$user_id;
$browser->get("https://webaccess.americafirst.com/servlet/pmservlet?tranType=login&".$content);
$browser->get("https://verifiedaccess.americafirst.com/passmark/signin.do?&detect=0");
$res = $browser->content;

# If this is the first time running from the current environment, the system will
# ask to answer a security question.
if ($res =~ m/not recognized/i) {
$browser->form_number(1);
$found_answer = 0;
for (keys %$secure_qa) {
if ($res =~ m/$_/i) {
$browser->field("answer", $secure_qa->{$_});
$found_answer++;
}
}
die $res."\nCannot register computer" unless $found_answer > 0;
$browser->field("doBind", "true");
$browser->submit();
}

# Check the pass phrase & enter the PIN
unless ($browser->content =~ m/$pass_phrase/i) {
die $browser->content."\n Incorrect security phrase.";
}
$browser->form(1);
$browser->field("PARAM_PASSWORD", $user_pin);
$browser->submit();

# Now we're logged in, so let's snag some info from the screen for our email
$account_summary = parse_summary($browser->content);
$email_summary .= "BANK SUMMARY:\n\n".summarize($account_summary)."\n=============\n";

$browser->get($account_summary->{Deposit}{Checking}{link});
$email_summary .= "LAST 20 TRANSACTIONS:\n\n".parse_checking($browser->content);

# Now, let's send the email...
$sender = new Mail::Sender;
$sender->MailMsg({
smtp => $smtp_server,
auth => 'LOGIN',
authid => $smtp_user,
authpwd => $smtp_pwd,
from => $email_from,
to => $email_to,
cc => $email_cc_list,
subject => 'Bank Summary',
msg => $email_summary,
});
exit;

sub parse_summary {
my $page = shift;
my $data = {};
$page =~ s/.+<\!-- Deposit processing -->//si;
$page =~ s/\<\/BLOCKQUOTE\>.+$//si;
$page =~ s/\*//g;
my @block = split(/<\!-- \w* processing -->/, $page);
foreach my $chunk (@block) {
my $type = "";
$chunk =~ s/.+<table[\w\s\"\=]*\>+//si;
$chunk =~ s/<\/table.*$//si;
$chunk =~ s/<tr>//sig;
$chunk =~ s/<td[\w\s\"\=]*\>//sig;
$chunk =~ s/\ \;//sig;
my @rows = split(/<\/tr>/, $chunk);
my $i = 0;
foreach my $row (@rows) {
my @rowdata = split(/<\/td>/, $row);
if ($i == 0) {
($type) = $rowdata[0] =~ m/\s?(\w*) Accounts/i;
} else {
my ($link) = $rowdata[0] =~ m/HREF=\"(.*)\"+/i;
my ($acc) = $rowdata[0] =~ m/\>(.*)<+/i;
$acc =~ s/<\/U>//i;
my ($bal) = $rowdata[1] =~ m/([\d\,]*\.+\d{2})/;
$bal =~ s/[^\d\.]//g;
$data->{$type}{$acc}{link} = $link;
$data->{$type}{$acc}{balance} = $bal;
}
$i++;
}
}
return $data;
}

sub summarize {
my $data = shift;
my $summary = "";
foreach $type (sort(keys %$data)) {
$summary .= uc($type)." Accounts:\n";
$summary .= "-----------------\n";
foreach $acc (keys %{$data->{$type}}) {
next if $acc =~ m/^\s?$/;
$summary .= "$acc\t\t$data->{$type}{$acc}{balance}\n";
$data->{$type}{total} += $data->{$type}{$acc}{balance};
}
$summary .= "-----------------\n";
$summary .= "total:\t\t$data->{$type}{total}\n\n";
}
return $summary;
}

sub parse_checking {
my $page = shift;
$checking = "";
$page =~ s/.+<\!-- Register Information -->//si;
$page =~ s/.+<input type=hidden name=ChkImgURL value="\S*\"+\>+//si;
$page =~ s/\<\/BLOCKQUOTE\>.+$//s;
$page =~ s/\<\/table\>.+$//si;
$page =~ s/\ \;//ig;
$page =~ s/<tr>//ig;
$page =~ s/style=\"font-weight\: bold\;\"//ig;
$page =~ s/\s{2,}/ /ig;
$page =~ s/<td CLASS=\"\w*\"( nowrap)?\>//ig;
$page =~ s/\r\n|\r|\n//g;
my @rows = split(/<\/tr>/, $page);
my $i = 0;
foreach (@rows) {
$i++;
my @values = split(/<\/td>/, $_);
next if !$values[0] && !$values[2] && !$values[3];
$values[0] =~ s/\s*//g;
$values[3] =~ s/\$//g;
push(@data, $record);
$checking .= "$values[0]\t$values[2]\t$values[3]\n";
last if $i >= 20;
}
return $checking;
}