Bulk Rename Files on Mac OSX

With the need to rename lots of files in bulk I have come to love a terminal application called ‘rename’. On Mac you can install it using Brew by running

brew install rename

Then if you want to rename all the files in a directory so they have a hyphen instead of an underscore you just do

rename -n ‘s/_/-/’ *

The -n flag says to run it as a test and don’t actually change the files. Once I confirm that it is working, I run it again without the test flag like so

rename -v ‘s/_/-/’ *

The v flag outputs what files were changed. You can find a few more examples at http://tips.webdesign10.com/how-to-bulk-rename-files-in-linux-in-the-terminal

Save And Load Bash Profile From Dropbox

With using different machines and installing a fresh copy of the latest Mac OS every year, I am constantly loosing my .profile. The .profile is a file used to save settings, shortcuts, and more awesomeness for your command line use in the Terminal app on Mac OS. I wanted to save my profile file to dropbox so I could use the same one across multiple machines and have it backed up. Here is how I did it.

  1. Move or create your profile file in Dropbox where you want it. I put mine at ~/Dropbox/Apps/Terminal/profile.txt.
  2. Make sure you don’t have a .profile file in your home folder so we can create a soft link to the dropbox version.
  3. Open Terminal and run the command to create a soft link to the dropbox version of the .profile. Your command may be different but will probably look something like
ln -s ~/Dropbox/Apps/Terminal/profile.txt ~/.profile

 

Now when you open a new terminal window, so the new profile is loaded, you will be using the profile from Dropbox instead of your home folder. Now in your home directory if you list out your files, you should see the soft link for the .profile pointing to the Dropbox version. Here is what mine looks like:

bash profile linked to dropbox location

Installing statsd on Ubuntu Server 10.04

Installing StatsD on an Ubuntu server (10.04) was surprisingly easy. Here are the steps that I took. These will work from a fresh install of Ubuntu server 10.04 or after having installed and started graphite.

# INSTALL NODE.JS
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install python-software-properties git-core
sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs

# CLONE THE STATSD PROJECT
git clone https://github.com/etsy/statsd.git

# CREATE A CONFIG FILE
cd statsd
cp exampleConfig.js dConfig.js
# edit config file your settings
sudo vim dConfig.js
node stats.js dConfig.js

# START STATSD
node stats.js dConfig.js

When you start StatsD, it will probably give you a warning like

(node) process.compile should not be used. Use require(‘vm’).runInThisContext instead.

but it is still working as expected so you can ignore it.

You can then test to make sure StatsD is working. Edit and run this simple PHP script to throw some stats at your StatsD.

 "$time|ms"), $sampleRate);
    }
    public static function increment($stats, $sampleRate=1) {
        StatsD::updateStats($stats, 1, $sampleRate);
    }
    public static function decrement($stats, $sampleRate=1) {
        StatsD::updateStats($stats, -1, $sampleRate);
    }
    public static function updateStats($stats, $delta=1, $sampleRate=1) {
        if (!is_array($stats)) { $stats = array($stats); }
        $data = array();
        foreach($stats as $stat) {
            $data[$stat] = "$delta|c";
        }
        StatsD::send($data, $sampleRate);
    }
    public static function send($data, $sampleRate=1) {
        $sampledData = array();
        if ($sampleRate < 1) {
            foreach ($data as $stat => $value) {
                if ((mt_rand() / mt_getrandmax()) <= $sampleRate) {
                    $sampledData[$stat] = "$value|@$sampleRate";
                }
            }
        } else {
            $sampledData = $data;
        }
        if (empty($sampledData)) { return; }
        try {
            $host = [your graphite host here];
            $port = [your graphite port here 8125?];
            $fp = fsockopen("udp://$host", $port, $errno, $errstr);
            if (! $fp) { return; }
            foreach ($sampledData as $stat => $value) {
                fwrite($fp, "$stat:$value");
            }
            fclose($fp);
        } catch (Exception $e) {
        }
    }
}

?>

Be sure to edit the $host and $port settings towards the bottom with the appropriate graphite settings.

When you run this script, you can run a tcpdump on the receiving machine to see if the UDP packets are getting to the machine.

sudo tcpdump -tn port 8125

If your StatsD is installed on a different machine/IP than Graphite, you can also watch StatsD send of the data by running

sudo tcpdump -tn port 8125 or port 2003

This is especially helpful if you have not properly set up your Amazon EC2 security group settings. Be sure to open a UDP port for the traffic coming through.

How-to Install Graphite on Ubuntu

UPDATED TO WORK WITH GRAPHITE 0.9.9!!! (10/13/2011)

The new release of Graphite 0.9.9 includes an experimental Flot JavaScript graph. Awesome!

Steps to installing Graphite on Ubuntu -> gist.github.com
Ubuntu Graphite install screencast -> youtube.com

 If you are looking for a how-to on graphite one (CAD) or statsd this is not the place.

A few months ago I blogged abou how much I love stats. One of the things that I shared in that post was a blog post done by the etsy developer team about statsd and graphite to track everything.

This week I was able to get graphite setup on a dedicated Ubuntu 10.04 on my Mac using VMware Fusion. I thought I would share the steps I took so others might be able to save some time in their setup. In setting up apache I just copied over the default virtual host settings since graphite will be the only thing running on the server. If you are planning on having multiple virtual hosts on the machine then you will want to configure the virtual hosts differently.

####################################
# BASIC REQUIREMENTS
# http://graphite.wikidot.com/installation
# http://geek.michaelgrace.org/2011/09/how-to-install-graphite-on-ubuntu/
# Last tested & updated 10/13/2011
####################################

sudo apt-get update
sudo apt-get upgrade

wget http://launchpad.net/graphite/0.9/0.9.9/+download/graphite-web-0.9.9.tar.gz
wget http://launchpad.net/graphite/0.9/0.9.9/+download/carbon-0.9.9.tar.gz
wget http://launchpad.net/graphite/0.9/0.9.9/+download/whisper-0.9.9.tar.gz
tar -zxvf graphite-web-0.9.9.tar.gz
tar -zxvf carbon-0.9.9.tar.gz
tar -zxvf whisper-0.9.9.tar.gz
mv graphite-web-0.9.9 graphite
mv carbon-0.9.9 carbon
mv whisper-0.9.9 whisper
rm carbon-0.9.9.tar.gz
rm graphite-web-0.9.9.tar.gz
rm whisper-0.9.9.tar.gz
sudo apt-get install --assume-yes apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common libapr1 libaprutil1 libaprutil1-dbd-sqlite3 python3.1 libpython3.1 python3.1-minimal libapache2-mod-wsgi libaprutil1-ldap memcached python-cairo-dev python-django python-ldap python-memcache python-pysqlite2 sqlite3 erlang-os-mon erlang-snmp rabbitmq-server bzr expect ssh libapache2-mod-python python-setuptools
sudo easy_install django-tagging

####################################
# INSTALL WHISPER
####################################

cd ~/whisper
sudo python setup.py install

####################################
# INSTALL CARBON
####################################

cd ~/carbon
sudo python setup.py install
# CONFIGURE CARBON
####################
cd /opt/graphite/conf
sudo cp carbon.conf.example carbon.conf
sudo cp storage-schemas.conf.example storage-schemas.conf
sudo vim storage-schemas.conf
### edited storage-schemas.conf to be the following
[stats]
priority = 110
pattern = .*
retentions = 10:2160,60:10080,600:262974
###

####################################
# CONFIGURE GRAPHITE (webapp)
####################################

cd ~/graphite
sudo python check-dependencies.py
sudo python setup.py install

# CONFIGURE APACHE
###################
cd ~/graphite/examples
sudo cp example-graphite-vhost.conf /etc/apache2/sites-available/default
sudo cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi
sudo vim /etc/apache2/sites-available/default
# moved 'WSGIImportScript /opt/gr..' to right before virtual host since it gave me an error saying
# WSGIImportScript cannot occur within  section
# if this path does not exist make it!!!!!!
# /etc/httpd/wsgi
sudo mkdir /etc/httpd
sudo mkdir /etc/httpd/wsgi
sudo /etc/init.d/apache2 reload

####################################
# INITIAL DATABASE CREATION
####################################
cd /opt/graphite/webapp/graphite/
sudo python manage.py syncdb
# follow prompts to setup django admin user
sudo chown -R www-data:www-data /opt/graphite/storage/
sudo /etc/init.d/apache2 restart
cd /opt/graphite/webapp/graphite
sudo cp local_settings.py.example local_settings.py

####################################
# START CARBON
####################################
cd /opt/graphite/
sudo ./bin/carbon-cache.py start

####################################
# SEND DATA TO GRAPHITE
####################################
cd ~/graphite/examples
sudo chmod +x example-client.py
# [optional] edit example-client.py to report data faster
# sudo vim example-client.py
sudo ./example-client.py

Original installation instructions -> http://graphite.wikidot.com/installation

Looking to run an Ubuntu instance on EC2? Check out http://uec-images.ubuntu.com/releases/10.04/release/

Now you are ready to install StatsD!