Facebook Group Search Web App

I have been working on my OpenSprinkler mobile web application and decided to port the new installer I developed over to my Facebook application. This allows an easier deployment of the web app. I also wanted to discuss recent changes that have occurred.

Facebook pushed new changes to the FB SQL methods that changed how comments were polled. These changes broke my older code but the code has been updated to support the current system. Also, I wanted to share some screenshots of the fully working application from my setup something I neglected to do in the past.

The code is available from Github. Also on Github are the instructions which are simply grabbing the files and placing them on your web directory.

The application auto detects mobile vs. desktop and redirects accordingly. Below are the mobile screenshots and further down are the desktop screenshots.

iOS Simulator Screen shot Jun 11, 2013 7.37.07 PM
The loading screen is displayed above when the app is first opened as a full screen application. This is installed via Safari on any recent iOS version.

iOS Simulator Screen shot Jun 11, 2013 7.37.10 PM
This is the initial screen the user is presented which gives a large search input field as well as some navigation options in the footer.

iOS Simulator Screen shot Jun 11, 2013 7.37.19 PM iOS Simulator Screen shot Jun 11, 2013 7.37.23 PM iOS Simulator Screen shot Jun 11, 2013 7.37.26 PM
These screenshots show the results page which is injected after a search. The first two show the posts and comments results by scrolling up and down. The blue boxes indicate a post which had a relevant comment. Tapping any post with comments pulls up the whole thread as shown in the 3rd screenshot.

iOS Simulator Screen shot Jun 11, 2013 7.37.35 PM iOS Simulator Screen shot Jun 11, 2013 7.37.59 PM
These two screenshots show the maps feature of the application. These use natural language recognition to parse the Facebook posts and geolocate them on the map. The filter box on the top allows quick filtering and the posts can be tapped to reveal the associated text.

iOS Simulator Screen shot Jun 11, 2013 7.38.03 PM
This is the about screen as well as version history and allows feedback to be sent to the developer if a user wishes.

Now for some desktop screenshots. The material is very similar to above so just the shots will be posted:

Rossie Rotation Advice Rossie Rotation Advice 2 Rossie Rotation Advice 3 Untitled

OpenSprinkler with Custom Web App (Updated)

I recently purchased an OpenSprinkler from Ray’s Hobby and am very impressed thus far! The device has accomplished its primary task of moving the controls away from a central box in the garage to any networked device (iPhone, iPad, laptop, etc.). It comes provided with a wonderful web interface out of the box. The install was very easy! I already had a Hunter irrigation control system and simply unplugged everything from it and moved over the connections to the OpenSprinkler. I even used the housing of the old system to house the new system. Below is a picture of the final physical install:

IMG_0656 IMG_0657

There are plenty of instructions available online on how to use the provided web interface but I have decided to make my own web app that is more mobile device friendly (iPhone and iPad). I decided to use jQuery Mobile and used the same boilerplate that I have used for my other web apps. The app is completed and released on Github.

All of the programs settings come directly from the OpenSprinkler HTTP interface so no need to re-enter all your settings again.

Discussion for the web app development can be found on Ray’s forums.

For assistance understanding how features work/mean please refer to the documentation on Ray’s website. Everything in this web app is based on his interface but simply adapted for the mobile interface.

Below are some screenshots:

iOS Simulator Screen shot Jul 27, 2013 5.48.29 PM iOS Simulator Screen shot Jul 27, 2013 5.48.49 PM

On first run, the app notices this is a new install and will assist in configuring everything! If everything is entered correctly you should see a success message similar to the one above. If an error occurs a message similar to the one below will appear to help you figure out what might be wrong

iOS Simulator Screen shot Jun 11, 2013 10.02.05 AM

iOS Simulator Screen shot Jun 8, 2013 11.27.14 PM

After the configuration setup you will be presented with this log in screen

iOS Simulator Screen shot Jul 18, 2013 6.36.17 PM iOS Simulator Screen shot Jul 18, 2013 6.36.32 PM

This is the home screen which gives the main options available to the sprinkler system. The weather is updated from Yahoo using the location set on OpenSprinkler.

iOS Simulator Screen shot Jul 2, 2013 10.26.18 PM

This is the side panel with various options including exporting/importing the configuration, all programs, station names, etc.

iOS Simulator Screen shot Aug 1, 2013 6.54.45 PM iOS Simulator Screen shot Aug 1, 2013 6.52.25 PM iOS Simulator Screen shot Aug 1, 2013 6.54.39 PM

This is the settings page which allows you to change device settings and edit the settings on the OpenSprinkler. Omitted from this list are two things though: the http port and the password. This is to prevent a lockout from the device

iOS Simulator Screen shot Jul 23, 2013 6.24.41 PM

This is the current status window which shows the device information as well as the stations. The background color behind each station signifies if that station is currently on/off. The program running is displayed underneath each station along with the scheduled or remaining time.

iOS Simulator Screen shot Jul 2, 2013 10.29.37 PM iOS Simulator Screen shot Jul 2, 2013 10.30.53 PM

This is the manual control screen which shows all the zones and allows a tap on/off for each zone. A green color after a zone has been tapped indicating that station is active

iOS Simulator Screen shot Jul 31, 2013 8.40.23 PM

The run once program selector. Each zone is shown with its own range slider

iOS Simulator Screen shot Jul 2, 2013 10.56.03 PM

This is the rain delay page which allows you to set a manual or automatic delay

iOS Simulator Screen shot Jul 27, 2013 5.39.39 PM iOS Simulator Screen shot Jul 27, 2013 5.39.53 PM

This is the log viewer that is inspired by David B. Gustavson script and in fact uses almost the same code but reflowed jQuery Mobile

iOS Simulator Screen shot Jun 9, 2013 10.56.29 PM iOS Simulator Screen shot Jun 9, 2013 11.00.37 PM iOS Simulator Screen shot Jul 27, 2013 5.55.42 PM

The main screen inside the programs section. Programs are expandable to display/edit settings. Scrolling down from program 1 (from above) shows the remainder of the settings available. The switch on the top between weekly/interval automatically shows/hides the appropriate settings. Also notice a slider selector for the duration (making this input easy!). The start/end time are also appropriately labeled in iOS to make time selection very easy as shown below along with the add new program screen

iOS Simulator Screen shot Jul 27, 2013 5.56.50 PM iOS Simulator Screen shot Jul 27, 2013 5.57.01 PM iOS Simulator Screen shot Jul 27, 2013 5.57.07 PM

iOS Simulator Screen shot Jul 2, 2013 10.46.37 PM

The program preview page which plots all of the interpreted programs on a timeline

Configuring WCCPv2 with a Dynamic IP

Setting up squid on Debian is fairly straight forward and my previous post explored that topic in depth. This time I want to share how I configured squid to communicate with my ASA 5505 which has a dynamic IP address. If you followed the earlier suggested guide you will notice it requires the ASA’s outside IP address. For a dynamic IP this would require editing the configuration each time. To avoid this I did the following:

Tell Debian to automatically load module ip_gre on boot by adding this line to ‘/etc/modules':


On Debian instruct networking to launch the script wccp2 after the NIC is up:

iface eth0 inet static
        post-up /etc/network/if-up.d/wccp2

Save the following to ‘/etc/network/if-up.d/wccp2′:


EXT=$(curl -s http://myip.dnsomatic.com/)

modprobe ip_gre
ip tunnel add wccp0 mode gre remote $EXT local dev eth0

ifconfig wccp0 netmask up
echo 0 >/proc/sys/net/ipv4/conf/wccp0/rp_filter
echo 0 >/proc/sys/net/ipv4/conf/eth0/rp_filter
echo 1 >/proc/sys/net/ipv4/ip_forward

iptables -t nat -A PREROUTING -i wccp0 -p tcp --dport 80 -j REDIRECT --to-port 3128
iptables -t nat -A POSTROUTING -j MASQUERADE

At this point you have the required tunnels built and you will notice the script grabs the internet facing IP using curl. If you do not have curl installed this will fail.

To get curl:

apt-get install curl

Finally, configure squid for wccpv2 by adding the following lines to ‘/etc/squid3/squid.conf':

wccp2_forwarding_method gre
wccp2_return_method gre
wccp2_service standard 0

On the ASA you can monitor the status of wccp using the following command:

sh wccp

Sample Output:

ciscoasa# sh wccp

Global WCCP information:
    Router information:
	Router Identifier:                   xxx.xxx.xxx.xxx
	Protocol Version:                    2.0

    Service Identifier: web-cache
	Number of Cache Engines:             1
	Number of routers:                   1
	Total Packets Redirected:            5879091
	Redirect access-list:                wccp_redirect
	Total Connections Denied Redirect:   0
	Total Packets Unassigned:            40
	Group access-list:                   -none-
	Total Messages Denied to Group:      0
	Total Authentication failures:       0
	Total Bypassed Packets Received:     0

The number of cache engines should be 1 if squid is properly connected. If squid is connected but doesn’t appear to be working it means your GRE tunnel is not setup correctly. Check your IPs at this point.

Upgrading to Squid 3.3 on Debian 6 (Squeeze)

The current Debian package for squid is on the 3.1.x branch and hasn’t upgraded to 3.3 yet which offers full HTTP/1.1 support. I already had a working squid on the Debian package and decided to just upgrade the running install. Below are the steps that led to a successful upgrade (backup your configuration before attempting).

Download and extract the source for squid 3.3:

wget http://www.squid-cache.org/Versions/v3/3.3/squid-3.3.4.tar.gz
tar xvzf squid-3.3.4.tar.gz
cd squid-3.3.4
apt-get build-dep squid3

Configure squid using the Debian environment:

./configure --build=i486-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --libexecdir=${prefix}/lib/squid3 --disable-maintainer-mode --disable-dependency-tracking --disable-silent-rules --srcdir=. --datadir=/usr/share/squid3 --sysconfdir=/etc/squid3 --mandir=/usr/share/man --with-cppunit-basedir=/usr --enable-inline --enable-async-io=8 --enable-storeio=ufs,aufs,diskd --enable-removal-policies=lru,heap --enable-delay-pools --enable-cache-digests --enable-underscores --enable-icap-client --enable-follow-x-forwarded-for --enable-auth --enable-basic-auth-helpers="LDAP,MSNT,NCSA,PAM,SASL,SMB,YP,DB,POP3,getpwnam,squid_radius_auth,multi-domain-NTLM" --enable-ntlm-auth-helpers="smb_lm" --enable-digest-auth-helpers="ldap,password" --enable-negotiate-auth-helpers=squid_kerb_auth --enable-external-acl-helpers=ip_user,ldap_group,session,unix_group,wbinfo_group --enable-arp-acl --enable-esi --disable-translation --with-logdir=/var/log/squid3 --with-pidfile=/var/run/squid3.pid --with-filedescriptors=65536 --with-large-files --with-default-user=proxy --enable-linux-netfilter build_alias=i486-linux-gnu CFLAGS="-g -O2 -g -Wall -O2" LDFLAGS= CPPFLAGS= CXXFLAGS="-g -O2 -g -Wall -O2"

Stop squid if it is running:

/etc/init.d/squid3 stop

Build and install the new squid:

make install

Move the binary to squid3:

mv /usr/sbin/squid3 /usr/sbin/squid3_old
mv /usr/sbin/squid /usr/sbin/squid3

If you have the following line in your current configuration you will have to remove it:

acl manager proto cache_object

Now you should be able to launch squid back up using 3.3.4

/etc/init.d/squid3 start

Crash Course: Cisco ASA 5505 Setup with QoS

I embarked on securing my home network while providing reliable VPN access. I had limited experience with Cisco’s IOS in the past so I was willing to make an attempt at taming an ASA (Adaptive Security Appliance) 5505. Of course my budget was limited so I opted for the base license (10 hosts). This is a limit I still do not quite understand. Some say its defined by ARP entries or XLAT’s. Honestly, I decided not to test the water.

I setup my network in a rather unique way. I kept my existing Airport Extreme with its current subnet and simply placed the entire router in a NAT on my ASA. This way my ASA will only ever see one host. This has some disadvantages but only in the configuration. For this example I will define the networks as (substitute yours in):

ASA Inside-Network =
ASA VPN-Network =
Airport Network =

It is important to note if this is your first time playing with an ASA it would be wise to attach the blue serial cable. It personally gave me the confidence to do anything I wanted without worrying about lossing the connection. I also setup my ASA completely via the CLI. I am no expert by any means, in fact I believe Cisco is currently emphasizing the ASDM however I just felt more comfortable in CLI. I do use the ASDM however I make it show me the commands before it sends them. This is for two reasons, one to teach me and two to prevent accidents.

I personally can attest to the vast amount of information on the internet about the ASA. This post is to help bring everything together into one place. I do not plan on explaining anything already available online. The configuration was done on ASA 9.0(2) but should work with anything 8.3 or higher (since the new NAT rules). These commands are expected to be issued on a factory default ASA.

The first thing you will need to do with your ASA is setup the basics (replacing the all-caps words):

hostname HOSTNAME
enable password PASSWORD

username USERNAME password PASSWORD privilege 15
aaa authentication http console LOCAL 
aaa authentication ssh console LOCAL 

http server enable
http inside
ssh inside

ssh version 2
ssh timeout 5
console timeout 5
management-access inside

clock timezone CST -6
clock summer-time CDT recurring

dns domain-lookup outside
dns server-group DefaultDNS

ntp server source outside prefer

Configure your VLAN addresses and setup DHCP server:

interface Vlan1
 ip address

dhcpd address inside
dhcpd dns interface inside
dhcpd option 3 ip interface inside
dhcpd enable inside

Example of opening a port through the firewall (you of course also have to forward it through the Airport):

object network SSH-Server

access-list outside_access_in extended permit tcp any object SSH-Server eq ssh

object network SSH-Server
 nat (inside,outside) static interface service tcp ssh ssh

access-group outside_access_in in interface outside

Note: The host is the Airport’s WAN IP. The address located on the subnet between the ASA and Router. This is because the Airport is performing NAT.

For the VPN portion of the configuration, I personally used the VPN wizard from the ASDM and it worked fine. Using OS X and iOS primarily, I chose to use IPSec. During the configuration I chose to do split tunneling which only redirects local traffic through the VPN and leaves the internet traffic through the client’s connection.

After you configure your VPN you might want to allow it to access the inside network:

object network inside-network

object network vpn-network

nat (inside,outside) source static inside-network inside-network destination static vpn-network vpn-network no-proxy-arp route-lookup

Block pings from hitting your outside interface with this:

icmp deny any outside

Enable scanning threat detection:

threat-detection scanning-threat shun except ip-address

To setup QoS first perform some speed tests at various times to gauge your actual bandwidth. You will also need to plan which traffic you wish to prioritize. We can only control what leaves our ASA. What comes in from your ISP has already traveled to its final destination and forcing the ASA to shape this traffic would be a waste. The argument says TCP will naturally kick in due to drop packets on the sender’s end and throttle down his send. I think it could have some advantages in paper however I have no real world evidence to support or refute anything nor do I plan on shaping the downstream traffic.

Here is a basic example of traffic shaping your outside interface’s upstream and prioritizing VoIP over IAX2:

class-map VoIP
 match port udp eq 4569

policy-map PriorityPol
 class VoIP

policy-map OutPol
 class class-default
  shape average 1800000
  service-policy PriorityPol

service-policy OutPol interface outside

Note: The numbers used were calculated based on a 2Mbit upload.

To setup SNMP use the following (fill in the community, location and contact):

snmp-server host inside poll community public version 2c
snmp-server location Home
snmp-server contact [email protected]
snmp-server enable traps snmp authentication linkup linkdown coldstart warmstart

To enable logging with email alerts perform the following:

logging from-address [email protected]
logging recipient-address [email protected] level critical

logging enable
asdm history enable
logging asdm informational
logging buffer-size 16384
logging timestamp

logging list email-notification level errors
logging list email-notification level notifications class auth
logging list email-notification level notifications class config
logging list email-notification level notifications class session
logging list email-notification message 109033
logging list email-notification message 109034
logging list email-notification message 315004
logging list email-notification message 315011
logging list email-notification message 605004
logging list email-notification message 710002
logging list email-notification message 111001
logging list email-notification message 111004
logging list email-notification message 113005
logging list email-notification message 113012
logging list email-notification message 113015
logging list email-notification message 611101
logging list email-notification message 611102
logging list email-notification message 102001
logging list email-notification message 199001
logging list email-notification message 199009
logging list email-notification message 502103
logging list email-notification message 111008
logging list email-notification message 613003
logging list email-notification message 603108
logging list email-notification message 719019
logging list email-notification message 719020
logging list email-notification message 719022
logging list email-notification message 719023
logging list email-notification message 111010

logging device-id ipaddress outside
logging monitor email-notification
logging buffered email-notification
logging mail email-notification
logging class auth mail warnings 
logging class config mail warnings 
logging class session mail warnings 
logging class vpdn mail warnings 

WCCP was something I quickly became a fan of and decided to jump right in. I am now running a dedicated Squid3 proxy VM that not only caches but also strips ads. By using the ASA’s feature and location I was able to transparently proxy the entire homes port 80 traffic. I decided not to touch SSL caching. The key to a proxy in THIS environment is the subnet. The proxy MUST be in the inside vlan subnet of the ASA (ex.

After setting up a working squid3 simply add these settings to your ASA:

access-list wccp_redirect extended permit tcp object inside-network any eq www 

wccp web-cache redirect-list wccp_redirect
wccp interface inside web-cache redirect in

If this still looks puzzling then you are probably missing the GRE tunnel needed. This is a great resource here: Cisco ASA and Squid with WCCP2.

Resizing a VMDK on ESXi 5.1 with GParted

When I first migrated from physical to virtual I didn’t put any research into my VM allocated HD space, after all it’s thin! Well, that was a giant mistake. I mistakenly filled my allocated space essentially converting a thin image to a thick image. To make matters worse I over-allocated the space triggering the VM to stop. Turns out, not surprisingly, ESXi needs some space on the drive for its own operation (if your datastore is on the same disk as ESXi). So, how do you get out of this sticky situation?

First, you need to re-thin your partition. To accomplish this task you will have to get the VM to boot up again. First you will need to regain a few GB of space on the datastore. The easiest way to do this is simply shutdown another VM that is running (if you can). This will turn off the allocated swap space. Now you will boot back into your over-allocated VM and delete the accidental files, if any. Then, run the following command to zero out the unused bytes of the disk:

cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill

If your using a Windows guest then you will need a utility called SDelete.

Shutdown the VM and login to your ESXi shell. Issue the following command to hole-punch your disk which converts your disk back into thin and deallocates all zero blocks.

vmkfstools -K /vmfs/volumes/datastore1/DebianVM/DebianVM.vmdk

Be sure to use the VM.vmdk and not the VM-flat.vmdk. After this is completed your free space will be reclaimed and you can boot any VMs you may have shut down. At this point we need to prevent this problem from ever occurring again. There are two approaches: permissions and/or resizing the partition and disk.

In linux, it’s possible a disk isn’t mounted where you expect it subsequently filling up the OS volume. To prevent this from happening chmod 000 the mount point before you mount your disks. The location inherits the permissions once mounted so the 000 will be ignored unless the mount is missing. Of course, this can only protect you so much. The worst case scenario, a VM being force offline, is stil possible because the VM still is over-allocated.

The next thing to do is resize the parition using a tool like GParted. This is an image I keep on a datastore because it is so useful. The idea will be to resize the partition down, copy the partition to a new vmdk that is properly sized (much smaller), reinstall the bootloader (grub), and enjoy!

There are a lot of guides on how to use GParted so I will just summarize the basic steps. First, open your vSphere client and add a new virtual disk to your VM on a datastore with space (such as an NFS disk). Make sure you appropriately size this one and select thin. Boot the system up using GParted. Find the original disk (ex. /dev/sda) and resize the partition down.

For a linux machine be sure to leave some space for swap. You do not need to copy the swap over, you can just recreate the partition on the new disk and issue a swapon once the VM is back online.

Once the disk is resized copy it over to your new disk (ex. /dev/sdb). Once completed reboot with a Debian iso (or any preferred disk for reinstalling the bootloader) and select rescue mode. After asking you some questions it will present you with a menu where you can select “Reinstall GRUB bootloader”. It will ask you the disk name so provide it the same disk as the one used in GParted for the copy destination.

Now you can shutdown your VM, disconnect your old vmdk, move the new vmdk to the same controller location as the old vmdk and boot your VM. If all goes to plan your VM should boot right back up with the smaller partition/disk.

I was able to execute the above steps, 100% remotely, using a Windows VM on the same ESXi host in under an hour. The resizing of the partition is the longest part but if your partition isn’t heavily fragmented it shouldn’t take too long. If you have your CD images on an easy to access datastore it will make the operation even quicker. After doing the exact steps a few times I estimate the entire GParted process takes 20 minutes (resizing the partition and copying the data to the new vmdk), the boot loader repair about 5 minutes, and playing with ESXi to switch the settings around a minute or two depending on your familiarity.

The alternative answer to this question posted all over the internet is using vCenter Converter. I installed the application but honestly it gave me too many issues opening my VM remotely and also wouldn’t open my offline VM locally citing “Unable to obtain hardware information” as the error. I decided not to spend much time learning a new tool that I would probably never use again. I was already familiar with GParted and since the partition has to be resized anyways (to avoid corruption to the filesystem) I decided it was the best approach to follow through with GParted all the way.

For the curious, there are methods which use vmkfstools to resize your disk instead of copying to a new virtual disk. These methods involve changing the header information in the .vmdk file to reflect the size you want. Then you use the vmkfstools to copy (thin) the old vmdk to the new. Since it will read the header info when making the new -flat it will generate the new size. Of course this would have to be done AFTER the filesystem has been resized, otherwise data loss is a risk.