Tag Archives: Raspberry Pi

USB Stick Fix for NTFS filesystem on Raspberry Pi

I’ve used USB sticks on the Raspberry Pi before, many times. So I plugged one in for some extra storage. It let me write once and then went into r/o mode. What the hell! Pulled it, plugged it into the PC, no problem. Plugged it into a Windows 7 machine, it complained about drive errors and fixed it. Try again, no luck. So I gave up for a while, it was full of Music that I play through the Sockso Music Server, so no big deal, but I really do want to use it for external storage as the SD card will fill up eventually. Plus I would like to use it as more storage for ftp as well and possibly for Sparkleshare which I intend to install.

One more look

It turns out that at some point while fiddling with the USB drive, I formatted it NTFS. I believe that I tried to format it ext4 and then ext2, neither worked. So I used a WIndows machine to make it NTFS, so it would work.

The issue is that the Rasp Pi does not come with NTFS support out of the box. So I needed to  install ntfsprogs to get it to work OK….

sudo apt-get install nfsprogs

Otherwise it immediately goes to read only. I have not tested but I assume nfsprogs also installs tools to check NTFS file-systems.

To mount my USB stick, I Need to execute …

sudo mount -t ntfs -o rw,uid=erick,gid=erick /dev/sda1 /media/sda/

Symlinks for Sockso Music Server

I have a symlink for music in my Music dir (/home/erick/Music. Set it to, main-collection -> /media/sda/Music/ , in this manner it points to the music collection on the usb stick and Sockso happily finds the music. Sockso does not like spaces in filenames!

 

 

Using mount with bind to access usb drive via vsFTP

I have a USB stick plugged into my Raspberry Pi for external storage, mostly to put music on for the Sockso Music Server to get at. But I wanted to use it a bit more for generic storage. FTP is great, you can get to it from any machine and the command line for it is the same on Win or Linux. So I can walk up to any machine, not have to install a thing and reach into a folder with FTP.

For instance, I have an infected Windows Machine, I don’t dare stick a USB stick in it. Instead I go to the command line, ftp to the Raspberry Pi and grab the tools I need from there.

The Issue

The issue was that I tried to symlink from the ftp directory to the USB drive. vsFTP will not follow symlinks for security reasons.

The Solution

Mount the directory you want under the FTP directory using bind. /media/sda is the USB stick mount point and the whole thing gets mounted under the FTP dir using…

sudo mount --bind /media/sda/ /home/ftpuser/usb-drive/

Resources

FTP on Raspberry Pi. An easy way to make shared folders

Alternatives to FTP

https://radu.cotescu.com/vsftpd-and-symbolic-links/

avconv replaces ffmpeg for Video/Audio Conversion

Recently I wanted to convert an AVI file to an m4v on Linux Mint 17.3. I tried to use ffmpeg, it’s not there. I can’t isntall it…hmmm. So I did some digging, it has been replaced by avconv. To get avconv….

sudo apt-get install libav-tools

To use

Online, I read that there are some syntax differences between ffmpeg and avconv, so I was reluctant to just symlink pointing ffmpeg to avconv and have scripts break.

But I did a quick test taking a good guess…

avconv -i timelapse-wx-station-4-08-to-7-12-2016.avi timelapse-wx-station-4-08-to-7-12-2016.m4v

…and, it worked. Got me a much smaller file than the AVI and worked it’s magic in a short minute or two.

Video of Weather Station from April 8,2016 to July 12,2016

This was what I was testing out with the conversion. Video taken by using fswebcam to gather a still shot of my weather station once per hour on the Raspberry Pi. Once per day the shots are rolled up into an AVI video and stored in a tmp folder which is mounted in RAM. Well the RAM folder does fill up after a while and I copied the file off and saved it on my desktop. This video shows the weather changing from spring to summer.

Resources

http://askubuntu.com/questions/432542/is-ffmpeg-missing-from-the-official-repositories-in-14-04

 

Sockso Music Server on Linux

The Sockso Music Server is very functional and quite easy to set up in standalone or daemon mode. It is cross platform as it only depends on a Java runtime environment being installed on the target computer.

Recently I loaded it on my desktop which runs Lubuntu 14.04. I tested it out on the desktop before loading it onto my Ubuntu  server PC, which holds my music repository.

  • I will outline installing the Java run time environment needed to run Sockso on an Ubuntu machine
  • The Sockso install procedure
  • Getting it to run as a daemon
  • Getting it to find your music
  • At the bottom of the page I will have some links to resources that I followed and will provide information for running Sockso on other platforms.

The Sockso install procedure

It is not so much an install like compiling/installing, apt-get or adding a package. It is a simple old school download and drop files in a directory install.

  1. Download the Sockso zip file.  You can do steps 2 and 3 while waiting for  the download!
  2. Create /usr/share/sockso directory as root or via sudo so all files are set to root:root. ( sudo mkdir /usr/share/sockso )
  3. Create Sockso data directory /var/sockso as root or via sudo. ( sudo mkdir /var/sockso )  If sockso is terminated uncleanly, the files in this directory can get corrupted and it will need to be rebuilt
  4. Extract the files to /usr/share/sockso/ ( sudo unzip sockso-1.5.3.zip -d /usr/share/sockso/ ) I am not 100% on my unzip usage, so this command actually made a sockso-1.5.3 folder under /usr/share/sockso. Then I needed to use sudo mv .. to move all the files and dirs up one level.

 

Install Java

On my server that runs headless I performed the following after I typed in java on the command line and it told me that it was missing. It usually resides at /usr/bin/java in a Debian/Ubuntu type of file system. If it is installed it will dump out a help file. Using the command which java will also tell you if it is installed…

The program 'java' can be found in the following packages:
 * default-jre
 * gcj-4.6-jre-headless
 * openjdk-6-jre-headless
 * gcj-4.5-jre-headless
 * openjdk-7-jre-headless
Try: sudo apt-get install <selected package>

I went for version 6 headless for starters. I am not sure what the difference between all the versions are, but version 6 worked for me.

erick@ubuntuserver:/tmp$ sudo apt-get install openjdk-6-jre-headless
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  ca-certificates-java icedtea-6-jre-cacao icedtea-6-jre-jamvm java-common
  libnspr4 libnss3 libnss3-1d openjdk-6-jre-lib tzdata-java
Suggested packages:
  default-jre equivs libnss-mdns sun-java6-fonts ttf-dejavu-extra
  fonts-ipafont-gothic fonts-ipafont-mincho ttf-wqy-microhei ttf-wqy-zenhei
  ttf-indic-fonts-core ttf-telugu-fonts ttf-oriya-fonts ttf-kannada-fonts
  ttf-bengali-fonts
The following NEW packages will be installed:
  ca-certificates-java icedtea-6-jre-cacao icedtea-6-jre-jamvm java-common
  libnspr4 libnss3 libnss3-1d openjdk-6-jre-headless openjdk-6-jre-lib
  tzdata-java
0 upgraded, 10 newly installed, 0 to remove and 5 not upgraded.
Need to get 44.2 MB of archives.

 

…and so on as it installed.

Reading Java Version

If you already have java and want to view the version…

java -version

…will get you the version, such as listed on my desktop PC…

erick@Precision-WorkStation-530-MT:/var/sockso$ java -version
java version "1.7.0_91"
OpenJDK Runtime Environment (IcedTea 2.6.3) (7u91-2.6.3-0ubuntu0.14.04.1)
OpenJDK Client VM (build 24.91-b01, mixed mode, sharing)

Test run

Before making it run as a daemon I wanted to test drive it. So the following command will start it up…

sudo sh /usr/share/sockso/linux.sh --nogui --datadir /var/sockso

When you terminate it, try to shut it down clean via a sigterm when you kill the process. I have read that killing it uncleanly can screw up the data directory ( /var/sockso ). Then you have to empty the directory and rebuild it’s contents. I haven’t had it screw up the directory yet.

Running Sockso as a daemon

Running Sockso as a daemon is an advantage when you are running on a server. It will startup when the machine starts and the machine will take care of closing it down cleanly upon shutdown.

Perl script for running sockso as a daemon

After moving the Sockso files to the proper location there will be a Perl file at /usr/share/sockso/scripts/init.d/sockso

Copy the sockso run file written in perl from…

 /usr/share/sockso/scripts/init.d/sockso

…to…

/etc/init.d/sockso

…using…

sudo cp /usr/share/sockso/scripts/init.d/sockso /etc/init.d/sockso

 

Edit the file and change the directory at the top of the file to point to where sockso is installed ( /usr/share/sockso ).

Also make it executable.

sudo nano /etc/init.d/sockso

sudo chmod +x /etc/init.d/sockso

Now that it is in the init.d directory, the following should work…

sockso (start|stop|restart)

Remember to change the directory at the top of the sockso to point to the /usr/share/sockso dir.

 

Starting Sockso on boot

Follow the Steps 4,5,6 on this blog post…

https://samiux.wordpress.com/2009/07/17/howto-sockso-1-1-8-music-server-on-ubuntu-9-04-server/

I have a copy here as a PDF –> sockso-start-on-boot , just in case the link above disappears.

 

Sockso Command Prompt

Sockso comes with it’s own command prompt to administer it. help will list the commands. You can use the Sockso command line to add music to Sockso’s collections, add and delete users and perform other maintenance to it.

There is also a management webpage where you can perform the same functions as via the command line.

Finding Music

There is a command line mode for sockso where you can point it to certain directories to index music from.

Run sockso to bring up it’s command line. At it;s command line use coladd and then the path to the folder that your music is in to add it. It takes a while to do this, it is indexing it into a database so be patient. You can add multiple directories into it’s collections. If you add music to a directory in the collection, sockso will find it and add it. By default it scans directories in it’s collections every 30 minutes. I’ve tested it and it is pretty cool, dump in some music and a little while later, it’s there like magic.

coladd /home/username/Music

collist will list all the collections. coldel deletes collections.

colscan will force a scan for new collections that have been added.

Symbolic Links to Music Folder

The sockso coladd command has issues with spaces in directory names. What I have done is made a bunch of symbolic links using ln -s directory of music directory-of-music. This makes it easy to see where all the music is and sockso just has to deal with my Music directory and if I add or remove music it will figure it out on it’s own. I show an example below in the Raspberry Pi section.

Sockso on Raspberry Pi

I just ( April 2016 ) installed Sockso on my Raspberry Pi. I got the idea of sticking a USB stick into one of it’s open ports and dump my music repository on it. Them with sockso I can get to it whenever I want. Previously I had it set up on my main server that I have to use Wake on LAN to start up when I am not at home. Having Sockso on the Rasp Pi allows me to get at it instantly and saves energy by not having to run a full fledged server just to play music remotely.

Below is a tree of the Music directory that I created under my home directory. As can be seen there are symlinks without spaces that point to locations on the usb stick, mounted at /media/sda.

erick@raspberrypi ~/Music $ tree -L 1
.
├── main-collection -> /media/sda/music
└── renee-ipod-music -> /media/sda/Renee's iPod/iTunes_Control/Music/

The USB stick is formatted it’s default way that it came, FAT32. I use pmount /dev/sda1 /media/sda to mount it. In this was it is mounted not as root, it is mounted by my user, so all files are easily accessed by my own user, locally and remotely using NFS or SSHFS. In this way I can add and remove files easily.

 

 Users

In Sockso there is a concept of users. You can have multiple people logged in and have personalized settings. You can even authorize uploads by setting that option.

Adding users at the Sockso command line works similar to adding users in Linux.

useradd NAME PASS EMAIL ISADMIN 1/0     Adds a new user

Commands:
userlist                                Lists the users
useradd NAME PASS EMAIL ISADMIN 1/0     Adds a new user
userdel ID                              Deletes a user
useradmin ID ISADMIN 1/0                Sets a user to be admin/non-admin
useractive ID ISACTIVE (1/0)            Toggles users between being active or not
coladd PATH                             Adds a folder to the collection
coldel PATH                             Removes a folder from the collection
collist                                 Lists the folders in the collection
colscan DIR (optional)                  Start a collection scan
propset NAME VALUE                      Sets a property
propdel NAME                            Deletes a property
proplist FILTER                         Lists properties
version                                 Show version information
exit                                    Exit Sockso


 

Resources

Where to get Sockso, it’s official site

http://sockso.pu-gh.com/

This site is a bit dated but still helpful.

https://samiux.wordpress.com/2009/07/17/howto-sockso-1-1-8-music-server-on-ubuntu-9-04-server/

Sockso Read Me

Requirements
————

Sockso should come packaged with everything it needs to run,
all you have to do is have Java installed on your computer.
You can download the latest Java version for free from
the Sun website at: http://www.java.com

To run Sockso under Windows just double click “Run Sockso”.
Easy!

“Linux”
——-

If you’re running Linux or something similiar then you may
just be able to double click the “linux.sh” shell script.
If this doesn’t work for you then you can run this script
from a terminal with:

$> sh linux.sh

Feedback
——–

If you’ve used Sockso then I’d love to hear what you think, so
please send me some email at: rod(at symbol)pu-gh(dot)com

Running as daemon
—————–
Usage: sockso (start|stop|restart)

 

 

 

 

 

sSMTP Installing and Configuration and Use Tips

Recently I was looking at creating a method of sending a warning email when ever my house temperature went below a threshold. I remembered that sSMTP was a simple way to send automated emails and CRON emails. I have some simple notes on what I did.

Installation

Very easy, just use apt-get from the command line…

sudo apt-get install ssmtp

Configuring

The configuration file (/etc/ssmtp/ssmtp.conf) can be edited using any test editor you typically use.

 

Config at /etc/ssmtp/ssmtp.conf

Below is my config file with the critical info blocked out. Lines in Red are what I modded to get ssmtp working for me.

The key pieces to get it working for me at least were…

hostname = My ISP’s domain

root = my complete email that I use at the ISP

mailhub = I looked it up in Thunderbird, it is the smtp.myispsdomain.net part.

AuthUser=my complete email that I use at the ISP. It might be different for you. Years ago it used to be just the user name part of email without the domain.

AuthPass = The password that goes along with my email.

I commented out the defaults for the ones that existed in the code.

The config file is a bit ugly after I touched it but I was trying to get this up and running quick and didn’t clean it up. But, hey it works!

 

#
 # Config file for sSMTP sendmail
 #
 # The person who gets all mail for userids < 1000
 # Make this empty to disable rewriting.
 #root=postmaster  <--- comment out
 
# The place where the mail goes. The actual machine name is required no
 # MX records are consulted. Commonly mailhosts are named mail.domain.com
  #mailhub=mail <-- comment out
 
# Where will the mail seem to come from?
 #rewriteDomain=
# The full hostname
 #hostname=raspberrypi <--- I was testing and kill this, failed to work
 # hostname has to be the mail domain! Or else it complains about
  # the raspberrypi part! The STMP server at frontier does that is.
  hostname=myispdomain.net
# Are users allowed to set their own From: address?
 # YES - Allow the user to specify their own From: address
 # NO - Use the system generated From: address
 #FromLineOverride=YES <-- Commented out and set below, I was testing!
# New Code put here 11302015
  root=me@myispdomain.net
  mailhub=smtp.myispdomain.net
AuthUser=me@myispdomain.net
AuthPass=myemailpassword
FromLineOverride=YES
#UseSTARTTLS=YES <-- Tried this, I didn't need it for my ISP.

CRON Email

Once installed if you or root on the machine have any CRON jobs, you will start to get email from them. You can stop this by appending …

> /dev/null 2>&1

to the end of the commands that are being run by CRON. Which will cut back on the emails that you will receive.

 Testing

I installed mail utils to allow sending simple messages…

sudo apt-get install mailutils

Then I sent a message via the command line…

echo "Test" | mail -s "Test Subject" me@myispsdomain.net

…and I was able to see it work OK.

Send files via email

If you want to send files you have to install mpack.

sudo apt-get install mpack

 

Then you can send files to your email like this…

mpack -s "Test" /tmp/web/log.txt me@myispsdomain.net

 Command Line Usage

If you execute ssmtp with an email address it will let you create an email from the command line. Which is good for quick emails to for example remind yourself of something, or send a snippet of code to yourself. You edit the email in the form of the example below and hit Ctrl-D when done and then it will send out.
ssmtp recipient_email@example.com
The following is an example right off the command line. Note the one line of space after the Subject, this is a must have…
erick@raspberrypi ~ $ ssmtp me@myispdomain.net
To:me@myispdomain.net
From:me@myispdomain.net
Subject:This is a test of ssmtp from the command line!

Hello there this is a test of the ssmtp from the command line tool. It could be used to send a reminder or small snips of code. Use Ctrl-D when you are done.

It is called up by using ssmtp emailtosendto@domain.com

Bye,
Me

Example of Sending CPU Temp Warning Emails

When I am away from home I can infer if my house is running to cold, which may indicate a problem with the furnace. The Raspberry Pi is light loaded, usually just idling, so the CPU temperature tracks the room temperature, with an offset. When I am away, I set the house thermostat at 47 degrees F. If it drops below this value the CPU temperature of the Raspberry Pi will drop below 34 degrees Celsius. So I can just have it send me an email if this happens. Then I can double check a log that is created of the temperature reading to see what is going on. Also I run a webcam pointed at an actual thermometer for a sanity check, this is logged by using fswebcam to take an hourly snapshot. So I have my bases covered for the most part. Obviously if the power is out, I am in the dark about the temperature, because the whole thing is down! Solving that is a future project.

Below is the snippet of code from a shell script that sits in /etc/cron.hourly that handles the warning emails that are sent to 2 addresses. variables mailaddr and mailaddr2.

temp is the CPU temperature in Celsius as an integer stripped using cut from the thermal_zone0 reading.

minimum and maximum are my temperature thresholds. I don’t care much about maximum but I have it set at 65 Deg. C. just in case.

# Read the temp and cut it to grab leftmost 2 characters, integer Temp
temp="`cat /sys/class/thermal/thermal_zone0/temp | cut -c1-2`"
#echo $temp

# Mail if about or below the limits
if (( $temp > $maximum )); then
   #echo "above"
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi HIGH CPU Temp > $maximum" $mailaddr
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi HIGH CPU Temp > $maximum" $mailaddr2

elif (( $temp < $minimum )); then
   #echo "below"
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi LOW CPU Temp < $minimum" $mailaddr
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi LOW CPU Temp < $minimum" $mailaddr2

fi

Boot Email

I want to know if an when the Raspberry Pi I run 24/7 ever reboots due to a power outage, so I have it send me an email. The line of code below handles it and is in the root crontab. I have it sleep for 180 seconds first, then send the email. This allows the cascaded routers which I have the Pi connected to and the cable modem, time to come on line.

@reboot sleep 180 && echo "Rasp Pi Rebooted" | mail -s "Rasp Pi Reboot!" me@myispsdomain.net

I also log boots in a file that I can view online, just to keep track in one record.

@reboot date >> /var/www/bootlog.txt

Keeping track of boots helps for instance if I am away from home and the power goes out. If I get the email that the Pi rebooted, I can check to see how long the power was down and what the temperature of the house is to see if all is well.

Every hour I take a time/date stamped webcam snapshot of a thermometer so I can just look to see how many are missing and have a rough estimate of how long the power was out and how cold the house got and verify that it is getting warmer because the furnace is on!

In the future I will connect a BME280 sensor to the Raspberry Pi that will be able to read ambient room temperature directly, along with humidity and barometric pressure. So I won’t have to infer the house temperature via the CPU temperature.

Resources

This is the page I used to configure ssmtp on the Rasp Pi.

http://www.raspberry-projects.com/pi/software_utilities/email/ssmtp-to-send-emails

Screenshot - 02162016 - 09:32:30 PM

Users, Groups and Sudo

One thing that I did once I got my Raspberry Pi up and running is to add a user account, other than the pi account that is there by default. I have an erick account on my machines, so why not have one on the pi.

useradd

So under the default pi prompt I used the useradd command to add erick as a user. I figured that I would not login as pi and gave pi a strong password.

useradd -m erick

This will prompt for a password and make a user directory under homes by default. It also fills the directory with files and directories based on the /etc/skel directory.

Lots of Options for useradd

Usage: useradd [options] LOGIN

Options:
-b, –base-dir BASE_DIR       base directory for the home directory of the
new account
-c, –comment COMMENT         GECOS field of the new account
-d, –home-dir HOME_DIR       home directory of the new account
-D, –defaults                print or change default useradd configuration
-e, –expiredate EXPIRE_DATE  expiration date of the new account
-f, –inactive INACTIVE       password inactivity period of the new account
-g, –gid GROUP               name or ID of the primary group of the new
account
-G, –groups GROUPS           list of supplementary groups of the new
account
-h, –help                    display this help message and exit
-k, –skel SKEL_DIR           use this alternative skeleton directory
-K, –key KEY=VALUE           override /etc/login.defs defaults
-l, –no-log-init             do not add the user to the lastlog and
faillog databases
-m, –create-home             create the user’s home directory
-M, –no-create-home          do not create the user’s home directory
-N, –no-user-group           do not create a group with the same name as
the user
-o, –non-unique              allow to create users with duplicate
(non-unique) UID
-p, –password PASSWORD       encrypted password of the new account
-r, –system                  create a system account
-s, –shell SHELL             login shell of the new account
-u, –uid UID                 user ID of the new account
-U, –user-group              create a group with the same name as the user
-Z, –selinux-user SEUSER     use a specific SEUSER for the SELinux user mapping

 

Once I was able to log in under my new account, I tried setting up fswebcam to collect some timelapse video and then I had my first hitch. I needed to be part of the video group to run fswebcam.

id

The id command ran from the command line…

id username

…will list not only the user and group ID of the user UID and GID. But all of the groups that he user belongs to. It has the options -u, -g, -G. -u lists the UID alone, -g is the users GID alone and -G lists all the group ID’s that the user belongs to.

usermod

I was not part of the video group so I would have to add myself, but I was not part of the admin group either so I was not able to even run sudo.

So with a quick logout to the pi user, I was able to add myself to the admin group.

sudo usermod -a -G admin erick

The sudoers file

If you run sudo visudo, it will open the /etc/sudoers.tmp file. At the bottom of this file there is a line that explains that accounts added to the admin group are allowed to run sudo.

# Members of the admin group may gain root privileges
 %admin ALL=(ALL) ALL

Now that I can sudo from my own account, I can login back in as erick and run…

sudo usermod -a -G video erick

…to add myself to the video group. Now I was off and running with using fswebcam under my account.

NFS and Users

With users there is the notion of the name and then there is the numerical UID. NFS uses the numerical UID to map across machines. If you plan on using NFS on multiple machines, it pays to keep the UID’s lined up between them. For example, if you set up 2 Linux machines from scratch, there will be a user at UID 1000, that would be you, whatever you called it by name. The first user is at 1000. If you use NFS to mount a directory from one machine to another, no problem it all lines up. The user at UID 1000 is the same on both machines, permissions work out, files can be moved back and forth, no problems.

But if like with the example of the Raspberry Pi above. User pi is created on the NOOBS Disk when you load the Raspberian option. It is at UID 1000 and GID 1000. So if you add and other user for yourself, guess what it appears at UID 1001. Something to keep in mind when using NFS. You can use NFS in a way that will get around this using the methods laid out in the NFS post.

But it is much easier to try to keep all of the name and UID’s lined up from the beginning and not have to worry about the trickiness business. Even it means adding a user to the Raspberry Pi and then moving the UID of the pi user to some other UID and yourself to UID 1000, GID 1000 if that will line it up with your other machines on the network.

Simple Linux Performance Benchmarking

Recently I did some very simple benchmarking of the CPU and disk drives on a few of my Linux PC’s and a Raspberry Pi. This was a quick test to see how all of the machines compare.

CPU: I went for some very simple tests of the CPU performance under a load, calculating Pi. The code is not the best at calculating pi, it is just there to exercise the processor(s) and provide some standard piece of executable code that can be tried on multiple machines. When the pi code runs it will use the processor at close to 100% when viewed using top.

13502 erick     20   0  2784 1964 1112 R  98.8  0.4   0:36.13 arm_pi

There are programs available that are better at calculating a lot of digits of pi, fast. An example is Hyper Pi for Windows that will certainly calculate Pi to many more digits in a given period of time than the pi shown in this post program can.

DISKS: To test disk speeds. I used a simple writing and reading out of the bitbucket ( /dev/null) to disk and noting the speeds displayed.


CPU Benchmarking

CPU benchmarking was done by running an executable that calculates Pi to an arbitrary number of digits. Below is the link to the C file which is easily compiled for the target machine. I downloaded this from the net, from this page… http://www.overclockers.com/forums/archive/index.php/t-402437.html

pi.c

For a Linux machine or the Raspberry Pi, downloading the file and executing…

gcc -o pi pi.c

or

gcc -o pi pi.c -lm

As the original post states. The -l means link a library. The m means the math library. I don’t think it is necessary???

…will create the executable pi.

To run in the directory it was create just call pi with an argument of the number of digits to calculate, for example Pi to 1000 digits is calculated via…

./pi 1000

The ./ in front of the program name is needed if you are executing the program from the directory that you are in, or else Linux will go off searching through the path for programs named pi. If you have created a bin directory under your home folder you can put pi, or any executable code there and run it from other directories on the machine by just calling the program without the ./

Run of Pi program for 1000 digits

Approximation of PI to 1000 digits

3.1415926535  8979323846  2643383279  5028841971  6939937510  5820974944
5923078164  0628620899  8628034825  3421170679  8214808651  3282306647
0938446095  5058223172  5359408128  4811174502  8410270193  8521105559
6446229489  5493038196  4428810975  6659334461  2847564823  3786783165
2712019091  4564856692  3460348610  4543266482  1339360726  0249141273
7245870066  0631558817  4881520920  9628292540  9171536436  7892590360
0113305305  4882046652  1384146951  9415116094  3305727036  5759591953
0921861173  8193261179  3105118548  0744623799  6274956735  1885752724
8912279381  8301194912  9833673362  4406566430  8602139494  6395224737
1907021798  6094370277  0539217176  2931767523  8467481846  7669405132
0005681271  4526356082  7785771342  7577896091  7363717872  1468440901
2249534301  4654958537  1050792279  6892589235  4201995611  2129021960
8640344181  5981362977  4771309960  5187072113  4999999837  2978049951
0597317328  1609631859  5024459455  3469083026  4252230825  3344685035
2619311881  7101000313  7838752886  5875332083  8142061717  7669147303
5982534904  2875546873  1159562863  8823537875  9375195778  1857780532
1712268066  1300192787  6611195909  2164201989

Calculations Completed!
Time: 1 seconds


It is possible to run this program to see how the various machines that you own and compare the CPU performance by using the pi program.

Minimal Boot

More advanced benchmarking of a machine would involve trying to boot using a disk that would allow only the command line and a minimal amount of background stuff to load. I might try this at some point to see what difference it makes. I would try to boot from a Ultimate Boot CD (UBCD) and then go into the mode that loads the minimal Linux boot and somehow get pi.c loaded via USB stick maybe? It would be an experiment!

For more advanced testing that runs outside of the OS it is possible to run the code included on the UBCD, for example the  for a any machine that will boot from CD. The CD contains a suite of benchmarking, testing and stress testing tools, in addition to other tools for working with a hard drive and unlocking machines.

Pi Benchmarking Script

For benchmarking I close out of all applications that are running and open one terminal to execute the pi program.

Code

Below is code for a script file that will run the pi program,looping it multiple times, calculating the results for 1000,2000,4000,8000,16000,32000,64000,128000 and 256000 digits. I wanted to see if there would be any noticeable variations in the time to calculate various amounts of digits among the machines. I didn’t notice much of a deviation when I lined up plots of the various machines. They were all nominally nearly a constant multiple of speeds across the multiple levels of pi calculation

The script creates two temporary files cols.txt that lists the number of digits it has run up to. Additionally a file called results.txt captures the amount of time that it took to calculate the corresponding number of digits.

#! /bin/bash
rm cols.txt
rm results.txt

DIR=/home/erick/pi
x=1000


while [ $x -le 300000 ]; do

echo $x
echo $x >> cols.txt

$DIR/pi $x > temp.txt
tail -n 1 temp.txt >> results.txt

x=$(( $x * 2 ))

done;

Results of the Pi Benchmarking Script

Raspberry Pi Model B, single core CPU at 700MHz
Time: 1 seconds
Time: 1 seconds
Time: 5 seconds
Time: 21 seconds
Time: 86 seconds
Time: 352 seconds
Time: 1438 seconds
Time: 5835 seconds
Time: 23507 seconds
Dell Dimension 530-mt Xeon dual core processor at 2.4GHz

The machine has 1GB RAM and I don’t think that comes into play running this program. The pi program is only running on one of the cores in this example.

Time: 0 seconds
Time: 0 seconds
Time: 1 seconds
Time: 3 seconds
Time: 14 seconds
Time: 58 seconds
Time: 232 seconds
Time: 932 seconds
Time: 3741 seconds
Dell Dimension 2400: Pentium 4 single core at 2.4GHz

Nearly identical performance as the Dell Dimension. This machine has 1.5GB RAM.

Time: 0 seconds
Time: 0 seconds
Time: 1 seconds
Time: 4 seconds
Time: 13 seconds
Time: 56 seconds
Time: 223 seconds
Time: 898 seconds
Time: 3596 seconds

Suprise

I benchmarked two older laptops, an old Dell Inspiron (2003) running Ubuntu 10.04 and a not so old Toshiba Satellite A135 (2009), Mint 17 xfce, both 1.6GHz processors.

Pentium M on the Dell and Celeron M on the Toshiba. 333 and 533 MHz busses respectively. I thought that they would be fairly similar in performance but was pleasantly surprised to find the Toshiba was a decent amount faster than all of the other machines! This was a machine that had only 512MB RAM and ran Vista poorly and I almost scrapped it. Until I bought another stick of RAM and loaded Linux Mint 17 on it! It is like a miracle how much better it runs. It is a good test machine to try out Mint as I might consider it for a future desktop machine.

Dell Inspiron

Time: 0 seconds
Time: 0 seconds
Time: 1 seconds
Time: 3 seconds
Time: 13 seconds
Time: 54 seconds
Time: 276 seconds
Time: 1134 seconds
Time: 4526 seconds

Toshiba Satellite

Time: 0 seconds
Time: 0 seconds
Time: 0 seconds
Time: 2 seconds
Time: 7 seconds
Time: 30 seconds
Time: 164 seconds
Time: 692 seconds
Time: 2747 seconds

A ratio of 1.82 times faster, not bad at all.

 


 

Disk Benchmarking

Disk benchmarking was done by writing from the /dev/null bitbucket to disk, flushing caches and then reading back a 1GB file, discarding it into /dev/null.

Write Script

The follow code is copied into a script file ending in .sh and made executable using chmod +x filename, will write 1GB of zeros to a file named after the of=.

#!/bin/bash
dd if=/dev/zero of=/home/erick/testfile-1024x1M bs=1M count=1024

Read Script

The following code is copied into a script file ending in .sh and made executable using chmod +x filename. This script will read back the file created by the write script, dumping it into the null device. It will read it back in 8k blocks.

#!/bin/bash
dd if=/home/erick/testfile-1024x1M of=/dev/null bs=8k

Flush Caches

In order to test the disk if the RAM on the the machine is sufficiently large and the write operation leaves the written data it in the cache, you need to flush the cache in order to have the machine actually read it from disk. Linux is pretty clever about using RAM that is not doing anything, not already in use for programs and OS, for a disk read/write cache. I remember in the old days of DOS that there were a number of these utility programs that could be loaded that would use some RAM as a cache, effectively Linux does the same thing natively.

Take the following code, copy into a file ending in .sh, such as flush-caches.sh and make executable via the chmod +x filename command and then run it between writing and reading the disk.

#!/bin/bash
sudo sh -c "sync && echo 3 > /proc/sys/vm/drop_caches"

Example Results

These disk write and read utilities can be used to test harddrives, USB sticks, SD cards, RAM Drives (effectively tests RAM speed) and so forth. It even can be used to test the network speed when a drive is mounted using NFS ( or using rsync, scp or sftp ) as this will usually be the speed constraint and not the drive R/W speed.

Dell Dimension 530-mt Primary Hard Drive

Write

erick@Precision-WorkStation-530-MT:~/bin$ ./harddrive-write-test.sh
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 24.0374 s, 44.7 MB/s

Read

erick@Precision-WorkStation-530-MT:~/bin$ ./harddrive-read-test-8k.sh

131072+0 records in

131072+0 records out

1073741824 bytes (1.1 GB) copied, 25.3427 s, 42.4 MB/s

Toshiba Satellite A135

Write

erick@erick-Satellite-A135 ~/bin $ ./harddrive-write-test.sh
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 36.6714 s, 29.3 MB/s

Read

erick@erick-Satellite-A135 ~/bin $ ./harddrive-read-test-8k.sh
131072+0 records in
131072+0 records out
1073741824 bytes (1.1 GB) copied, 35.093 s, 30.6 MB/s

 

Old, circa 1998 machine, 4GB primary Hard Drive (/dev/sda) and 8GB Compact Flash card (/dev/sdb) Secondary drive

The interesting thing here besides just how slow the speeds are, is the fact that the CF card is actually faster than the hard drive. The drive is actually pretty loud on this machine as well. When CRON runs at the top of the hour, there is definitely a small burst of sound, enough to serve as a reminder of the time if you are in the same room as the machine. When you SSH into it, you can hear it grind away for about 2 seconds as it reconciles the password! I used this machine for remote monitoring before I had a Raspberry Pi, works good enough for that with it’s limited RAM and HD space. Now it is just a backup in case the Pi is down.

Disk /dev/sda: 4303 MB, 4303272960 bytes

Write: 1073741824 bytes (1.1 GB) copied, 144.733 s, 7.4 MB/s

Read: 1073741824 bytes (1.1 GB) copied, 138.498 s, 7.8 MB/s

Disk /dev/sdb: 8195 MB, 8195604480 bytes

Write: 1073741824 bytes (1.1 GB) copied, 91.4438 s, 11.7 MB/s

Read: 1073741824 bytes (1.1 GB) copied, 73.3132 s, 14.6 MB/s

Raspberry Pi SD Card

Using a shorter test via the following script to write…

#!/bin/bash
dd if=/dev/zero of=/home/erick/testfile-10000x8k bs=8k count=10000

and to read…

#!/bin/bash
dd if=/home/erick/testfile-10000x8k of=/dev/null bs=8k
Write To SD Card
erick@raspberrypi ~/bin $ ./sd-write-test.sh
10000+0 records in
10000+0 records out
81920000 bytes (82 MB) copied, 8.45663 s, 9.7 MB/s
Read Back from SD Card
erick@raspberrypi ~/bin $ ./sd-read-test-8k.sh
 10000+0 records in
 10000+0 records out
 81920000 bytes (82 MB) copied, 4.38749 s, 18.7 MB/s
RAM Drive Performance

tmp on my Raspberry Pi is set up as a RAM Drive (RAMFS) on my Raspberry Pi. So this gives some indication of how fast the RAM can be wrote and read to.

Using the following write script…

#!/bin/bash
dd if=/dev/zero of=/tmp/testfile-1000x8k bs=8k count=1000

and read script…

#!/bin/bash
dd if=/tmp/testfile-1000x8k of=/dev/null bs=8k
Write to RAMFS
erick@raspberrypi ~/bin $ ./ram-write-test.sh
1000+0 records in
1000+0 records out
8192000 bytes (8.2 MB) copied, 0.0458338 s, 179 MB/s
Read from RAMFS
erick@raspberrypi ~/bin $ ./ram-read-test-8k.sh
1000+0 records in
1000+0 records out
8192000 bytes (8.2 MB) copied, 0.0339184 s, 242 MB/s

 

 

end

CPU Temperature Monitoring on the Raspberry Pi

One of my thoughts for using the Raspberry Pi is to monitor ambient temperature in my house and send me warnings if it is too low. This would be helpful when away from home in the winter and there is an issue with the heat that I might need to know about fairly quickly.

Eventually I might consider building in a weather station capability into the Raspberry Pi by using both an indoor temperature, humidity and barometric sensor and an outdoor temperature and humidity sensor. So with that in mind I also want to make some simple graphs of data every hour or so. Something that can be just saved into a text file and requires no extra graphing code on the client or the server.

CPU Temperature Experiment

I started out by experimenting with taking reading of the CPU temperature of the Raspberry Pi in order to get a little practice with some live data while I was awaiting the arrival of a Bosch BME280 ambient temp., humidity and barometric pressure sensor.

sSMTP

I plan on using sSMTP to send me warnings, such as when my ambient temperature falls below a critical value. I have already worked up some code to monitor the CPU temperature and email me if it goes out of a specific range, to get a bit of practice with coding this into a script. Plus it was a good way to try out sSMTP. sSMTP will be covered in more detail in a separate post.

Snippet of script that sends email if the upper and lower CPU temp limits are exceeded

The temperature read by the cat /sys/class/thermal/thermal_zone0/temp is in milliCelsius, so using cut on it will grab whole degree values. The rest of the code is pretty straightforward. The variables minimum and maximum are setup to be the appropriate values in whole degrees Celsius. mailaddr is exactly what it sounds like. What is nice is that it just gives you a very simple method of sending a basic email warning when the temperatures get out of bounds.

# Read the temp and cut it to grab leftmost 2 characters, integer Temp
temp="`cat /sys/class/thermal/thermal_zone0/temp | cut -c1-2`"

#echo $temp

# Mail if about or below the limits
if (( $temp > $maximum )); then
   #echo "above"
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi HIGH CPU Temp > $maximum" $mailaddr
elif (( $temp < $minimum )); then
   #echo "below"
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi LOW CPU Temp < $minimum" $mailaddr

fi

 


Simple Graphical Temperature Logging

To just monitor the CPU temperature on a hourly basis and have a simple graphical representation of it I had to do some digging online.

I dug around the web and found a bit of Perl code to make a simple ASCII graph of the temperatures that I was measuring for the CPU. I wanted to be able to simply post to the web a running capture of temperatures, grabbed hourly by using CRON. I did not want to have to rely on graphing tools either on the Raspberry Pi or the client computer to interpret data. I spent a while searching for something simple and this was about as easy as it can get. I found a one line, long line! Method for creating a simple graph using Perl. I looked around a lot on line and wanted to avoid anything that would have to be installed on either the Raspberry Pi or the client machine to be able to plot data. So after a long search this piece of Perl code is about as simple as it gets and will plot and ASCII based graph across one line each time it executes.

The key variables in the code are min, max and w. The variable w controls the width of the plot in characters that it will be allowed to reach when the input value is at max. The variables min and max control the minimum and maximum input values expected and scale the graph accordingly. PLOTTABLE_FILE is the path/filename of the file that the output will be sent to.

 PLOTTABLE_FILE=/var/www/tmp/cputemp-milli-celsius.txt

Perl Code Snippet

The code resides in a script in the /etc/cron.hourly directory and produces a file that is readable via the web, with a new entry every hour. The filename is cputemp, not the lack of a .sh extension. This is important when you create scripts that will reside in the “CRON” folders. It also has to be made executable by using sudo chmod +x filename.

(cat /sys/class/thermal/thermal_zone0/temp ) | perl -ne '$min=20000; $max=65000; $w=79; use POSIX; $d=ceil(log($max)/log(10)); $w-=$d; $v=$_<$min?0:$_>$max?$max:$_; $s=$w*$v/($max-$min); $bar=join("", ("*")x$s); $bar=~s/.$/|/ if $v==$max; print sprintf("%${d}d ",$_)."$bar\n";' >> $PLOTTABLE_FILE

Sample Output

Below of a capture of a few lines of output from the text file that is generated by the Perl code. It does not look as clean as it would by looking at the text file. So I have a capture of that here…. cputemp-milli-celsius

42236 *********************************************************************
41160 *******************************************************************
41160 *******************************************************************
41160 *******************************************************************
40084 *****************************************************************
41160 *******************************************************************
42774 **********************************************************************
44388 ************************************************************************
45464 **************************************************************************
45464 **************************************************************************
46002 ***************************************************************************

Alternate Method to Read CPU Temperture for the Raspberry Pi

There is another way to read the CPU in a more human readable form. Executing the line below…

/opt/vc/bin/vcgencmd measure_temp

…will show a result like this…

temp=45.5'C

Resources

Source of Perl Script

 

Raspberry Pi

Reduce writes to the Raspberry Pi SD card

After 5 months of solid up-time for my Raspberry Pi server, which has been running great. It has been taking a picture every hour and from them creating a creating a timelapse video every day. Also it is being used as a place to drop files to periodically from other place on the network, a little bit of file storage. Eventually I will add more storage space to it to use it even more for network storage.

Recently, I started to think about the potential wear of the SD card as I came across several articles online dealing with the topic. I decided to make a few changes to the Raspberry Pi configuration to reduce the amount of writing to the SD card.

Write Saving #1: Using a tmpfs

I editted /etc/default/tmpfs. In it the comments state  that /run, /run/lock and /run/shm are already mounted as tmpfs on the Pi by default. Which I have observed. This was a change made a while ago for the Pi according to the buzz online. I additionally set RAMTMP=Yes to add /tmp to the directories put on the tmpfs. This sets up access to /tmp with rwx-rwx-rwx permissions. There was a suggestion that I saw online to limit the sizes of the various directories, I added that as well.

# These were recommended by http://raspberrypi.stackexchange.com/questions/169/how-can-i-extend-the-life-of-my-sd-card
# 07262015, mods for using less of the SD card, RAM optimization.
TMPFS_SIZE=10%VM
RUN_SIZE=10M
LOCK_SIZE=5M
SHM_SIZE=10M
TMP_SIZE=25M

The OS and some programs will use /tmp. But so do I. I created a /tmp/web folder under it when the Raspberry Pi boots. Into this folder files go such as the hourly photo and the daily video that scripts create for the webcam that is attached. I have reduced 3 hourly writes to just one photo. I keep only one on the SD card as I don’t want to risk losing a bunch of them taken during the day if I totally relied on the tmpfs. If I was using a UPS, I would have no problem saving all of them on the tmpfs and occasionally backing up to the SD or another device. The big saver is the daily timelapse.avi for the web that is created daily from all of the hourly captured photos. It is many megs in size gets written daily and it doesn’t matter if I lose it. It can be recreated from the photos at will. So it is the perfect kind of file to throw on a RAM file system.

I also store the hourly and daily logs that I create using the cron driven logcreate script that I run. The logcreate script creates an hourly log that is concatenated into a daily log on the tmpfs then every day the daily log will concatenate into a full log, that is rotated, on the SD card, so I have a permanent record. Need to put the link for this here!

What is a tmpfs?

It is a RAM Disk, a.k.a. RAM Drive that allows RAM to be used as a hard drive. Obviously when the power goes out, it goes away. So we don’t want anything important to go there. But for things like files that I make such as the hourly photos that my Web Cam takes and the video it makes daily and logs, it is perfectly fine for usage. It is not a really big deal if the power went out and I lost this information as it will be recreated shortly anyways.

Caution

The only issue that I see with having logs on a tmpfs would be a situation where the Pi got in a state of weirdness where it started rebooting itself and then you had no logs to track down the problem. Then I suppose, it would be just a matter of changing the /etc/fstab file to revert to putting the logs back onto the SD card for a while to track down the problem. But, for a Raspberry Pi like mine that is running stable and I am not doing many experiments with right now, having the logs in volatile memory is not something I worry about. Plus it is easy to make a script to backup the logs to the SD card or another computer, if you manually reboot it, so you can save them if you like when you have control of the reboots.

Write Saving #2: Turning off swap

If the Raspberry Pi runs out of RAM, not likely if it is a server set up for light duty usage, it will start to use swap which is on the SD card, causing writes to the swap file. Mine rarely touches swap. I would rather tune the thing for better memory use than have it use swap.

It is possible to turn off swap usage using the command…

sudo dphys-swapfile swapoff

This is not persistant and needs to be done on every boot. It could be put into the root crontab by editting it using sudo crontab -e and adding the line. Or creating a script for it along with other items that are to be run at startup.

@boot dphys-swapfile swapoff

Online, people said that there was another way to turn it off by reducing the swap file size to zero, a config file for swap, can’t remember the name. But it is claimed that when it reboots it just overrides that and makes a default 100M swap file.

Write Saving #3: Moving /var/log to a tmpfs

One of the biggest offenders as far as writing to files periodically is the logs that live under /var/log and it’s sub-directories. You can create an entry in /etc/fstab that will create a tmpfs for /var/log. The only caution here is daemons, like Apache that require a directory to exist under /var/log or else they will not start. Apt also has a directory under /var/log, but it creates itself when apt runs for the first time so that is no problem. The apt directory has logs that keep track of what apt installs or uninstalls, good info to know about. News seems to work fine creating a directory for itself too. So for me only Apache is a problem.

  1.  Put an entry in /etc/fstab…
     tmpfs /var/log tmpfs defaults,noatime,mode=0755 0 0
  2.  Found out that news and apt folders create themselves when these things run.
  3. Apache is the one thing that does not like a missing folder so made a Kludge for now using ~/bin/setup-tmp.sh where I create /var/log/apache2 and chmod it 750. Then I restart apache using apachehup.sh, which just restarts it. Apache was failing to load when I pointed the log dir to /tmp in /etc/apache2/envvars under the export APACHE_LOG_DIR directive.

Write Saving #4: noatime

As you can see above one of the options used in the /etc/fstab file is the noatime option. By default the Raspberry Pi uses this option for the mount of the SD card. If you add mount points of your own to the card, make sure noatime is used. Without it Linux makes a small write each time a file is read to keep track of when it was last accessed, this obviously causes writes. It is possible to use it for the writes to the tmpfs as I am doing above. It saves a bit of time as the system does not have to do a write when a file is just being read.

Another good use of noatime is for drives connected across the network. For example on NFS mounts noatime is a really good choice. The network is generally slower than devices attached to a PC and having to send a write across every time a file is read, slows things down a bit when moving many files.

 


Been running this setup with the RAM savings for a few months now with no problems. I hardly ever see the ACT light blinking on the Pi anymore.

 

The LEDs have the following meanings :

  • ACT – D5 (Green) – SD Card Access
  • PWR – D6 (Red) – 3.3 V Power is present
  • FDX – D7 (Green) – Full Duplex (LAN) connected
  • LNK – D8(Green) – Link/Activity (LAN)
  • 100 – D9(Yellow) – 100Mbit (LAN) connected

From http://www.raspberrypi-spy.co.uk/2013/02/raspberry-pi-status-leds-explained/