Use sed to retreive links from a Firefox bookmark file

Sed is a great text manipulation tool. It allows to manipulate text through simple substitution commands.

It did occur to me that I had a few backup .json files in which Firefox bookmarks were stored. As I did not want to overwrite the existing bookmarks, but I wanted to retreive just a few links, I tried to open the file with a text editor.

The result was not pleasant:

I decided to use some CLI magic to transform this in a list of URLs which is easily readable, instead. It can be done in one line:

$ sed "s/\"uri\"\:\"/\\`echo -e '\n\r'`/g" bookmarks.json | sed "s/\"[^@]*$//" > new_bookmarks.txt

Where:   bookmarks.json -> bookmark file name
            new_bookmarks.txt -> new file name where the links will be saved

Enjoy!

MailMe: how to get notified automatically via email through BASH

In my job, I have to perform calculations on a remote machine to which I connect through ssh. It happens quite often that I have to run a few jobs overnight, if not for days. I could check manually for their termination, but I came up with a better way.

In a previous post I described how useful it can be to run script in background on the machine without having to be logged-in. Combining the power of screen with the utility of mail we can create a script which will automatically send an email whenever the job is finished.

The Script

#!/bin/bash
#
##
## This scripts sends an automated email as soon as a job is finished
##
## ***   Use with screen:
## ***  
## ***   $ screen -S [job-id]               (start screen)
## ***   $ mailme [job-id]                  (run mailme in screen)
## ***   Ctrl+a d                           (detach)
## ***   Ctrl+d                             (kill screen, from session)
## ***   $ screen -ls                       (check running screens)
## ***   $ screen -r [screen-id]            (resumes session)
## ***   screen -X -S [screen-id] kill      (kill screen, from outside)
## ***  

# Need argument: job id which needs to be checked for completion
if [ -z "${1}" ] ; then
    echo "Please insert job ID as argument"
    echo "Exiting status 1"
    exit 1
fi

# Need second argument: email
if [ -z "${2}" ] ; then
    echo "Please insert email as second argument"
    echo "Exiting status 1"
    exit 1
fi

# Put your email here
zzEMAIL="${2}"
zzJOBID="${1}"
zzFREQ=3600 #Frequency of checks, in ms
zzEND=true

# Loop that checks job status
while "${zzEND}"
do

    zzSTRING=$(bhist -d | grep "${zzJOBID}")
   
    if [ ! -z "${zzSTRING}" ] ; then
   
    zzEND=false
   
    echo -e "Dear Sir,\n\nThis is an automatic email from grep Linux indicating that the job ID\# ${zzJOBID} has been terminated (succesfully or not).\n\nThis is the output:\n\n $(bhist -d) \n\nBest, \ngrep Linux | mail -s "Job ${zzJOBID} Completed!" "${zzEMAIL}"
   
    else
   
    sleep "${zzFREQ}"
   
    fi
   
done

exit 0 

The comments should provide a satisfactory explanation, but it could be used straight away for jobs submitted through the routine bsub.

You would just need to run the script from a screen and the game is on. If you do not remember how to do this, I have put some reminders in the header section of the script, or you can check my guide to screen.

You can run multiple of these scripts in multiple screen sessions if more jobs need to be tracked.

Afterthoughts

This script has been designed to check job progress and send an email  whenever the job is complete. Through an easy modification of the condition to be checked in the variable zzSTRING in the loop, anything can trigger the delivery of an automated email, potentially making the scope of this script very broad.

Have a play with it and let me know how you use/modify it!

 

Motion: use your webcam as a security camera!

Last month, my landlord told me he had to send an engineer to fix the heating in my room. I could not be at home at the time they would come, but they would have access anyway.

This is one of the many occasions in which you wish you had a hidden security camera at hand (together with when you want to know if you roommate is really stealing your chocolate and when you suspect there are undesired animals in your room!).

But the solution is just in front of you, right now! Your computer's webcam is a hidden and usually unsuspectable object and it can be easily turned into a monitoring device.

And Linux helps us doing it very easily with a small program called motion.

Motion saves snapshots from your webcam whenever movement is detected by the webcam itself. Increasing the capture rate, it can produce decent videos as well.

Configure

Motion needs to be configured before it can be used. Configuration allows for a big list of customizations, as frame capture rate, image resolution of the frames, motion sensibility, webcam properties and many others. It is very easy as most of it is already configured.

First of all, copy the config file in your user account. From a terminal:

$ cd
$ mkdir .motion
$ sudo cp /etc/motion/motion.conf ./.motion

Then change the ownership of the file to your user (it was copied from root)

$ sudo chown [username] ./.motion/motion.conf

 And open it with your favourite text editor (which is emacs, of course):

$ emacs ./.motion.conf


In the ideal case, you would not need to modify anything. But it would be safe to skim through the file to see if there are things you want to change. Comments (line starting with #) are very exhaustive and it is difficult to get lost. The most important would be:

videodevice /dev/video0

which selects the webcam device used, or the directory where the snapshots are saved:

target_dir /home/user/your_folder

Usage

Funnily enough, using the program is easier than configuring it. Just open a terminal and run the command:

$ motion

You will see a lot of gibberish-looking lines and if you are in front of your webcam, you can test it straight away, as it will be saving pictures already, if you are moving.

To close the program when you are finished just press

ctrl-c

and you will not lose anything as the pictures will be saved in the folder you chose during the configuration stage.

Bonus

As an addition for extra security, you could connect the folder to the internet over cloud sharing services as dropbox or copy, and look for movements in your room real time! 

Screen: how to run processes in the background after log out

So, you want to run a very long process on your ssh server but you need to log out.

It sounds like an impossible problem, but as long the computer is on, why can't it run processes in the background? Well, it can, and there is a simple solution using a quite clever command-line window manager:

Screen

Screen can create multiple windows in a shell, from which you can "detach" and leave them running in the background, even if you log out of your user account.

Create a Screen

From a terminal, type:

$ screen -S [choose-a-window-name]

And you will immediately enter the window just created.
From this window you can type any command and leave it running in the background.

To detach from the window and go back to the terminal, just press the combination:

ctrl-a d

And you will go back to the terminal, where you can start more screens if needed, or even log out without stopping the job.

Access a previously created Screen

From a terminal, type:

$ screen -r [window-name]

to revert a previously created screen. If you do not remember the window name, you can list them through the command:

$ screen -ls

which gives information about the active screens and their names.

Kill a Screen

If your job is finished in one screen and you want to close it, you can go revert it and press:

ctrl-d

to close it, or you can close it from outside the screen through the command:

$ screen -X -S [window-name] kill

Conclusion

Screen is a very powerful tool and its uses span a much wider scope than the ones described here. It can be regarded as complete full-screen window manager.

For ssh users, though, it is a lifesaver as it allows to run for nasty and long processes at anytime of the day without needing your presence.