2 month hiatus

OK, I have not posted in 2 months. The type of work I do has increased as a result of many businesses having to reduce their staffs. So I have been working a bit of overtime. Also I have been investing time in upgrading my kitchen. But there has been some minor progress in building my skills in Python. I did write a enough of a program to automate OWFS to log temperature data. This is the graph of my first run. Can you spot the odd trend ?



Now while I live in a place where it does get over a 100 degrees in the Summer, over 120 degrees seems extreme. So I did some troubleshooting and I figured out the issue. The sensor for the outside was close to the garage door. The sensor was factoring in the heat reflected from the garage door. I did expect a spike in temperature in the afternoon as the sun directly shines on the door at that time. To prevent this extreme spike from future readings I moved the sensor a few feet over to the side of the door. I ran the program for a day and the logs no longer showed the extreme spike. So the next time I run the program I should have a better set of logs to analyze.

OWFS, GLPK, and Python

OWFS: Two weeks ago I installed OWFS and got it working with the sensors from the command line. This week I experimented with the python bindings to OWFS. Via Python I could easily connect to the sensors and read the temperature. What seemed difficult via the Python bindings was changing the temperature scale from the default of Celsius to common weather temperature of Fahrenheit. Via the command line this was an easy task, via Python binding it was a perplexing situation. After reading the documentation it appears the Python binding is not full featured enough to do this. So I figured I could write a simple Python line to issue the command line setting to complete this task. What prevent this was the binding to OWFS did not leave an obvious path to make this change and did not provide a command to find the path. So my conclusion was to write a Python script where half of it issues commands to the OS as if it were a shell script and then pull the data into Python logging and graphing. Below are some examples of what I needed I needed to do.

First I have to mount the file system for the controlling device. Since this is a USB device I need to do this as root. For now I just run my IDE Eric4 via gksudo until I can think of a better way

os.system('/opt/owfs/bin/owfs -u /var/1-wire/mnt')

So next I enumerate the interfaces on the wire

dirlist = os.popen('ls /var/1-wire/mnt/alarm/').read()
print "these are the interfaces"
print dirlist
PreIfaces = dirlist.split('\n')
IfaceLst = []
for listing in PreIfaces:
   if re.match('10\.\w', listing):
           print listing, "is a single interface"
           listing = str(listing)
           IfaceLst.append(listing)

So here is where I change the temperature scale and then see if my change took place

os.system('echo "F" > /var/1-wire/mnt/settings
/units/temperature_scale')
TempScale = os.system('cat /var/1-wire/mnt
/settings/units/temperature_scale && echo')
print TempScale

The rest of my code is learning the best (or most Pythonic) way to pass the list to the function os.popen so I can programmaticly get a temperature on all three interfaces without having to name each one. I think I will also just log the data to three text files using the CSV module for this current version. Then for version 2 of the script I may move logging to SQLite. I have the book
Beginning Python Visualization for guidance with making graphs in Python.

GLPK: So on the Ubuntu Linux machine I got PyMathProg installed. In my past post I stated I had the PyMathProg installed but that was the earlier version (since it was an Ubuntu package), that does not work with the examples on the current PyMathProg site: http://pymprog.sourceforge.net/. So tonight I uninstalled the Ubuntu package version and installed the version from the tarball I downloaded. Then I ran the basic PyMathProg examples and it seems to work now. So give another cheer for progress.

Various current status

OpenOpt: I was able to get OpenOpt to work. It can produce the expected results for the optimal solution for a problem where I already knew the answer. But OpenOpt has a limitation. The default solver does not produce a "sensitivity analysis" and the developer does not plan to add one. The optional solver may have one, but I would have to focus my energy in a different forum to find out. Since I have limits on my time, I think I should focus on a Linear Programming tool that is designed to what I want to accomplish. Hence I will move my focus in Open Source linear programming tools to GLPK.

Home Project: I live in an area of Southern California where it gets hot. My house has various hot spots. For some time I have been wanting to address these hot spots but I did not want to do so with guess work. Also for each fix, I wanted to know how much of an improvement each fix provided so I could determine some kind of return on investment. I found a solution. The open source program OWFS can read 1-wire network temperature sensors and log the data. I just finished configuring the OWFS with the hardware. I now have three separate sensors that report each sensor's temperature. When I am finished with configuring OWFS for my needs I will start a project where I will have one sensor in my garage (which faces west and gets very hot), one sensor in the part of the house above the garage, and one outside. More to come as I configure OWFS prepare for this project.

OWFS: Be sure to check out the examples section, especially the garden
Hobby Boards: 1-Wire USB Adaptor, Adaptor Cable RJ45 to RJ11, Temperature Sensor


Podcast of the week:
Normally the podcast "Bloomberg on the Economy" does not rank as one of my favorites. Often the host acts like a alumi insider by dropping names as if we all know who the players are. But once in a while he does have an interesting guest. This time the guest was Gillian Tett, an editor of the Financial Times. She was promoting her book "Fool's Gold: How the Bold Dream of a Small Tribe at J.P. Morgan Was Corrupted by Wall Street Greed and Unleashed a Catastrophe.''. As per the podcast she mentioned how the original group in J.P. Morgan modeled the credit derivatives. The part that was interesting was the original model revealed the problem we see today. So the original team took steps to prevent problems. When others in the industry learned of the models they learned how to implement the same investments make the money but did not implement the safeguards of the original team.

This is following a theme I have been learning lately. A good model is only useful in the long run when you understand it's limitations and flaws, then take steps/implement processes to address the limits and/or flaws. Hence the need for "sensitivity analysis" for models that support them and the need to examine models under great to terrible conditions.

Bloomberg On the Economy: Search for "Tett Discusses `Fools Gold,' Her Book About J.P.Morgan" on the page and download the mp3 file.

Econometricians are also sensitive

One of my favorite podcasts is EconTalk.While last weeks edition featured Ed Leamer from UCLA. Most of the podcast was a discussion about Macroeconomics within thescope of Ed's latest book, the host (Russ Roberts) gave Ed an opportunity to discuss econometrics. Ed made a point that any econometric study should include a sensitivity analysis to see verify if the study is robust. If the study is too sensitive then the study should be considered but not seen as absolute.

This compliments an earlier post on sensitivity analysis for Linear Programming. The lesson I am learning is that while each technique of data modeling can guide organizations to the best answer, it is also important to understand under what conditions does the result remain valid. Without understanding the sensitivity of the model and its results, the effort to make a better decision with scientific skills, could be wasted given the skills may not be applied within the proper conditions. Model building for decisions is not complete until a sensitivity analysis and applying it's results has been performed.

Listen to the EconTalk podcast

Ubuntu: New software packages installed

As a follow up to my previous post, I just installed GNU Linear Programming Kit (GLPK) and PyMathProg. GLPK is recommended by Larry of http://industrialengineertools.blogspot.com/ and PyMathProg is a Python implementation of GLPK. I also installed SimPy which is a Python queuing simulation modeler. At some point, after I build a core set of skills, I will provide samples of theses programs in the blog.

Ubuntu to you too or me in this case.

With a focus on Open Source tools while working on Operations Management skills (or the more popular term seems to be Operations Research) I needed to get a machine on an Open Source OS. A various times in the past I have had both physical and virtual machines running Linuxes such as Debian, Red Hat, CentOS, Gentoo and BSDs such FreeBSD (which I really like, but that's a post for another time) and OpenBSD.

Last weekend I decided to try Ubuntu Linux 9.04. Since it is based on Debian I figured it can't be too different. The only sense of difference was when it came to the GUI, it would take me a few tries to get x-window working properly. On Ubuntu it was simple, simple simple. The only thing that took a little bit of thinking was applying the wireless driver. The wireless card (on an old laptop) I used only had Windows drivers. So I got to try the famous Linux NDIS wrappers to run Windows drivers on the Linux OS. Of course since the network was not enabled yet, I had to download the .deb package files, burn them to CD, then mount the CD. Since a command line is not immediately available in Ubuntu by default, running dkpg or apt-get was not an option to install the local .deb files for NDIS. But I found that if I right click the .deb file, "GDebi Package Installer" will install the .deb file (yes I was clued in enough to know to check http://packages.ubuntu.com/ for dependencies first and download them). So once NDIS wrappers was installed, I ran it against the Windows drivers, then entered the wireless info and the rest is wireless network success.

I have heard good things about Ubuntu for awhile. Now that I have installed it, I am impressed with it as a easy to install laptop Linux. I plan to install Operations Research tools on Ubuntu laptop to expand my skills. Do you have any success stories with Ubuntu?

Hmmm, here's to wishing AIG had Wargammers

I listen to a number of Podcasts. TED Talks, EconTalk, Stuff you missed in History Class, FLOSS Weekly, etc. There is one podcast that overall has not been very interesting since it seems like each episode is mainly a book review. But of it's collection of book reviews, there was one that stood out from the rest. The podcast is named "The Invisible Hand". The specifc podcast that caught my interest was the book, Wargaming for Leaders: Strategic Decision Making from the Battlefield to the Boardroom. While this podcast was generally interesting since it described how the consulting firm Booz Allen Hamilton ran simulations in a gaming style for each business case, there was a specific example that makes you slap your forhead and cringe. The client for Booz Allen Hamilton asked for various cases that included real estate as a factor. For fun the Booz Allen Hamilton consultant modeled the banking industry's loans and risk transfer with insurance and it's impact if housing prices dropped. As per the model built, the whole thing collapsed as soon as housing prices dropped by $1. What was the client's reaction ? "Good thing that will never happen".

Yikes !! I wonder if AIG had wargammers talking with the actuaries if we the US tax payers would have 80% ownership of AIG today ? Well, it's all an academic excersise now. Have any of you used a simulation to prevent any problems ?

The Invisible Hand podcast web site
MP3 file for Wargaming for Leaders...