All posts by joet3ch_admin

Alternative Tornado Logging

Tornado uses the standard logging library, by default, and sends logs to STDOUT. Sometimes you may want logs stored in a database or flat-file.

Logging to flat-files

This method is fairly simple. You could pass the ‘log_file_prefix’ parameter via the command-line, or more elegantly add the options in your code:

If you have multiple instances of the same app running, I would keep the logs separate by using the port number:

Storing logs in a MongoDB collection

MongoLog is a really cool open-source centralized logging module for Python and MongoDB. It’s available on Github at https://github.com/andreisavu/mongodb-log.

Follow the installation docs in the README to get MongoLog all setup. After cloning and installing MongoLog, we need to modify our Tornado app.

First, import MongoLog:

Next we will override the ‘Application.log_request’ method to implement MongoLog.

After everything is up and running, you can view the raw logs in the database or use the web ui that ships with MongoLog:

MongoDB-Centralized-Logging2

Hurricane Irene – Testing Our Infrastructure

The past 24 hours have been filled with a lot of media hype, some rain, some wind, and a few darwin awards. In my home at the Jersey Shore we still have a ‘situation’ — no electricity.

Luckily we are served well by Twitter and can find out storm details by fellow local internet users, thank you Twitter.

Internet access has remained available, thanks to the awesome local ISPs and UPS units on our modems.

Amazon Web Services seemed to have no outages during this storm, thank you AWS.

Thank you also to AT&T and Verizon for keeping our cellular services online. I’m posting this via a tethered iPhone.

However the electricity infrastructure has proved to be unable to handle single points of failure. We are on day 2 of no power, thanks to a broken transformer miles away.

1 – Why don’t we have high-availability redundant infrastructure?

2 – Why don’t we have safer, less vulnerable, underground power lines?

We need to develop a redundant system like the internet for our electric. I won’t hold my breath. For now I’ll take it into my own hands and outfit my place with solar and diesel backup solutions.

Thank you Irene for testing our infrastructure.

Update: I just ordered this generator so I can keep writing code during power outages.

Yamaha EF2000iS 2,000 Watt 79cc OHV 4-Stroke Gas Powered Portable Inverter Generator (CARB Compliant)

My Dream Apple Machine

An extra-wide touchpad and dual detachable iPad displays on a MacBook Pro. Wow! Add 32GB of RAM and quad SSD RAID too. I’m not sure why they left the optical drive in this mockup, that has got to go. Lets hope Apple is really brewing this, I will camp out on the street for this one!

NewImage

My Workstation and Network Infrastructure

This is where I spend most of my time …

DSCN1611

My primary workstation is a MacBook Pro 17” with i7 CPU, 8GB RAM, 256GB SSD and a 27” Apple Cinema Display. I use an Apple bluetooth keyboard and Magic Trackpad.

I also use a MacBook Pro 15” for testing security related stuff that I don’t want to pollute my main workstation with.

DSCN1615

In the background of my view are 2 Samsung displays. On the bottom is a 52” and above is a 24” that I use for displaying network monitoring and analytics info. Below the monitors is a Mac Mini running Lion Server, a Drobo with 8TB of storage, an Apple Airport Extreme, APC UPS and my dusty Xbox.

DSCN1626

Lets dissect my network infrastructure…

Dual internet connections with 1 DOCSIS v3.0 modem and 1 DOCSIS v2.0 modem.

Protecting my network are 2 security appliances, a Cisco ASA 5505 and a custom built pfsense box.

The core is a HP ProCurve 1810G-24 switch that sits on top of my retired Cisco 3500 WS-C3548-XL-EN Switch.

My primary wireless network is powered by an Apple Aiport Extreme, and I run a Buffalo WHR-HP-G300N router with OpenWRT for guests.

Most of my servers are EC2 instances running on Amazon Web Services. However I do use a Dell PowerEdge 1950 server with VMWare ESXi for local testing.

Also in the rack is a Samsung 17” CRT Monitor, 2 Belkin KVM switches, a Foscam FI8918W IP camera and a Belkin UPS.

Static Websites

It’s been over a decade since I touched a static website. After Amazon announced website endpoint support for S3, I wanted to give it a try since the benefits are pretty appealing.

A static website eliminates the databases and serverside code execution.

Hosting a static site on Amazon S3 eliminates supporting the web server, load balancing, caching, etc.

By switching from a common cms system such as WordPress, your most likely to gain a lot of speed and lower the possibility of security vulnerabilities.

A con for some users is that your only option for increasing functionality is to write JavaScript and let it run clientside. For a blog, this is perfect in my opinion.

The tools I used to create a full featured static blog:

Cyrax generates the static site (python / Jinja2 templates)

Amazon S3 serves the HTML and JavaScript

Flickr for photo hosting (could do this with S3, but I like the iPhoto integration with Flickr)

Feedburner serves the rss and publishes new posts to Twitter via Feedburner Socialize

Disqus handles the user commenting system

Google Analytics takes care of traffic analysis

In the end, I am sticking with WordPress for T3CH.com. For simpler blogs, I’d definitely consider using Cyrax.

WordPress App for iOS Devices

Testing out the official WordPress app for iOS.

So far, I think there is some primary functionality lacking… There is no way to save a draft post. And the post editor is very simple.

I attempted to paste an image into this post with no luck.

I do value the simplicity but would like to be able to save drafts and embed images in the posts from my iPad and iPhone.

As I am typing this last line, I can’t even see it since the post editor does not scroll up. I had to rotate into portrait mode to finish this. I hope there are more WordPress apps to choose from!

After Upgrading to Mac OS Lion (workstation and server)

I upgraded 2 machines to Mac OS Lion yesterday. The download is not fast and the file is almost 4GB. So it‘s a good idea to save a copy of the installer app after it finishes downloading. That file will be auto-deleted after the Lion upgrade. Then you‘ll be able to create usb installer disks and upgrade other machines while saving bandwidth.

First up was the MacBook Pro 17 (my primary workstation). The upgrade was seamless and I have no complaints. I disabled Spotlight indexing since it was lagging the system, now it’s super snappy.

Next I upgraded a Mac Mini (my primary server). The Mac Mini was at a different physical location, so I performed the upgrade via Apple Remote Desktop. Since the box was already running Mac OS 10.6 Server, the App Store forced me to purchase both Mac OS Lion and the new Server app… which I expected.

 

Mac OS Lion Server Upgrade CompleteMac OS Lion Server Upgrade Complete

 

 

 

The server upgrade did take a bit more effort. Here are the issues that I had to deal with:

– I was charged for Mac OS Lion upgrade twice. I emailed the App Store and received a refund less than 1 hour later.

 

Mac OS Lion Server purchaseMac OS Lion Server purchase

 

– WebDAV is broken. The settings are there (they have been moved from the Web Server config screen to the File Sharing config), but it just doesn’t function… I tested with cadaver and it would not work. Instead of spending the whole night banging my head on this issue, I setup WebDAV with Apache in a Linux VM.

– FTP server is gone. Yeah FTP isn’t ideal, but I do have some devices which only support offloading files via FTP. This bugs me, but I’ll use vsFTPd in a Linux VM instead.

– The config options are lacking. In 10.6 there were way more settings and configurable options available in the Server Manager app. Here is an example of the Web Server settings gui:

 

Mac OS Lion Web Server Settings

Before You Upgrade To Mac OS Lion

Mac OS Lion was officially released earlier this week and I have been holding off on the upgrade since I didn’t want to disturb any of the projects that I was working on. Well now the weekend is here and I am ready to give it a spin! FWIW, I have been running developer beta releases of Lion on a spare MacBookPro but now I want the final release on my production workstation.

Before you upgrade, it’s a MUST to check the RoaringApps App Compatibility List and compare all of your apps to see which ones will work and which ones will not. This list is built by the community and is the most critical tool in determining if your ready to upgrade.

roaring_apps

Another MUST is to create a clone of your existing system, I use SuperDuper!. If your upgrade fails, SuperDuper makes it simple to roll back the system state quickly.

SuperDuper!

Ok, I am ready to go upgrade my box… And then will be upgrading my Mac OS Servers!