Internet connection troubleshooting

So your internet connection is not working or it is really slow. It is pretty common and Linux has a big number of tool to try to pinpoint the problem.

The first easy steps that really help understand what is wrong in the connection do not actually require any hack or software. I will call them Safety Checks. It includes parts of the routine checks they ask you to go through at annoying helplines and - for how annoying they could seem - they solve the problem most of the times. You could go straight to the command line and try find a solution without any pre-check, but why complicate life so much, especially when you are already frustrated by a non-working internet connection.

This guide is for ethernet connections (I will add wireless later, so switch to cables if you are using wireless, as often wireless has channel, signal or authentication problems).

1. Safety Checks (these apply to any OS)

a. Check again that cables are connected correctly, that your router is on and it is not giving problems (e.g. error messages through flashing leds). If the hardware is giving problem, do not think you will manage to magically fix your internet with a string of command.

b. If everything is ok, shut off your modem and wait a few minutes before turning it on. Meanwhile, reboot your computer as well (I know it is very annoying!). You will thank me later if that solves the problem.

c. If you tried that already/it is not enough, it really helps to try internet on other computers or operative systems. That will help localize the problem. If internet does work on other devices, you can be sure that the problem is within your device. This is good, as it is usually easier to solve (the problem depends on you - where you can put hands on - and not on them). If this is the case, you will be able to solve the problem, otherwise the problem could not be within your range (Power failures, remote server shut down, bad cabling, network congestion).

d. Control DHCP settings. DHCP is a tool in your router that assigns IP to each device automatically, making it non-static. You need to control DHCP is activated both on router and local device. You usually access the control panel of your router by typing the default route (Gateway, the IP of your router) into your web browser address bar. If you do not know what is it, it is usually 192.168.0.1 or 192.168.0.255 (you can try them), or more easily found under the name of Gateway in connection information (depends on OS).

e. Ping! Most of the times bad connectivity is caused by bad DNS servers. DNS are used to translate website names into IP numbers. Pinging the direct IP will then work as usual if the problem is in the DNS. In order to ping, run (from terminal or otherwise):

$ ping 74.125.132.147

That IP is actually from a Google server. If the problem is in the DNS, you will receive back using this command, but not using:

$ ping www.google.com

In which case you will need to change your DNS (depends on OS, but they are usually under your connection settings, together with IP, Gateway and Subnet mask settings). Try put the OpenDNS: 208.67.222.222 as primary DNS and  208.67.220.220 as secondary.

If both ping do not work, that could be due to invalid ping (rare), server down (rare), firewall blocking packets (quite rare) or most likely incorrect routing or missing connectivity.

2. Eliminating interface problems

Check that the local interface is working. It might have been shut down for some reasons or it might be broken. From a terminal, logged in as root, type:

# ifconfig

The output should include eth0 (and lo and wlan0 if you have a wireless connection). But we will care about eth0, being the cable connection. If the interface is on, you will see a line with

inet addr:192.168.0.4  Bcast:192.168.0.255  Mask:255.255.255.0

and it should say UP BROADCAST MULTICAST instead of BROADCAST MULTICAST. This is how the output should look like:

eth0      Link encap:Ethernet  HWaddr 00:1b:13:84:3g:4d 
          inet addr:192.168.0.7  Bcast:192.168.0.255  Mask:255.255.255.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:2924 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2287 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:180948 (176.7 Kb)  TX bytes:166377 (162.4 Kb)
          Interrupt:16

You should see no (or just a few) errors, dropped, overruns or collisions. Each could give a hint towards the problem, but if you do see errors (say more than 1%) you should be warned that it is usually hardware problems (either local but also possibly not within your powers), but I will deal with them in section 4.

If your interface is down, instead, run:

# ifconfig eth0 up

And check with ifconfig again. If internet does work now, problem solved! Otherwise, continue (I will assume the interface is active now and working, if you can't manage to turn it on, it might likely be a distro-related problem).

3.  Eliminating Routing Problems

Sometimes, regardless of what your OS might be showing (Connection established icons or messages), it does not understand that it is connected, but routed badly (missing IP, different Gateway, DHCP conflict). That can result in a bad connectivity.

a. Try this command:

# route

and you will see an output of this kind:

Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.0.0 * 255.255.255.0 U 0 0 0 eth0
link-local * 255.255.0.0 U 1000 0 0 eth0
default 192.168.0.1 0.0.0.0 UG 100 0 0 eth0

where the last line is the one that matters. You will see your Gateway (check it is correct) and the flag UG, with the interface you are using, eth0. If you do not, try to receive the information about your gateway and run:

# route add default gw 192.168.0.1 eth0

and you should see your gateway appear under the route command again.

b. Try the command:

# arp -a

 and you will see an output of this kind:

? (192.168.0.1) at 00:1b:13:84:3g:4d [ether] on eth0

where in brackets it will be your Gateway. If you see this, instead:

? (192.168.0.1) at <incomplete> on eth0

There are problem, and most likely it will be a wrong setting for gateway, or DHCP conflict. Check you DHCP settings in the router again, as pointed in Section 1.d. If that does not work, continue.

c. Type:

# cat  /etc/network/interfaces

to show your /etc/network/interfaces file. It should look like:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

If instead of the last line, you have:

iface eth0 inet static
       address 192.168.0.2
       netmask 255.255.255.0
       gateway 192.168.0.1

it means that you are using a manual setting on your computer, so replace it with "iface eth0 inet dhcp" or configure IP settings correctly, making sure that all the addresses matches both on router and machine, and that DHCP is switched off on the router.

This should solve the routing problems and you should have a well connected router to your pc. If the internet connection is still not working, go to the next section.

4. Heavy duty tests

Before this, you should have solved your problem already, and if you are here, it means that something big is going on.

Run this commands:

# mii-tool -v

# ethtool eth0

# ethtool -S eth0

# netstat -i

They should give different outputs, but with visible errors, if there are any.
I will sum up the meaning of those errors:

Collisions: they usually happen within normal connection below 0.1% of the times. Poorly terminated cables or badly working networking card can increase such errors.

CRC errors: file transfer completed but with corrupted chunks in the transit. The presence of CRC errors usually hints toward electrical noise, such as damaged cable or non-properly connected cables.

FIFO and Overrun errors: this is usually a sign of excessive traffic

Length errors: this is most frequently due to incompatible duplex settings.

Carrier errors: faulty interfaces or networking equipment.

If you report such errors and/or still cannot manage your connection to work, it might be a problem which is not in your powers to solve. Try call your ISP.

5. Speed test

Your internet connection might be working now, but you want to test its speed. There are loads of online in-browser services to do that, but what I hate about them is the fancier and fancier flash interface they propose which literally slows down my not so powerful computer.

For this reason, I prefer to just see the speed of a file transfer and judge by myself the speed (in comfortable KB/MB [kilo/mega-byte] notation, not the annoying Mb [mega-bit], which is mainly used by big companies to confuse people about broadband speed). The way I do it is by using wget:

$ wget --output-document=/dev/null http://speedtest.wdc01.softla\

yer.com/downloads/test500.zip

This command will download a test file (no worries, redirected to /dev/null, so nowhere basically) and you will see the speed of download on screen. You can stop it anytime with Ctrl+C.


Hope this helped to solve your broadband problems or at least give you a better insight into it.

Quickly create new empty files on Gnome (Nautilus) desktops

I don't know if you, like me, use the Desktop as a place for urgent notes. I do, and a lot!

I create empty files and rename it with the note I want to write in it and dispose of them easily deleting them whenever I don't want the note to appear.

Unfortunately the way to create those files is not one of the nicest, if you are in a rush: right-click on Desktop, find Create Document and click on it, find Empty file and click on it.

That is why I found a quicker way using the keyboard:

- Right click on Desktop
- Press D and then Return

And you have a new empty file ready to be written!

The Matrix effect

It has been a while since last time I wrote a blog post. I planned ahead of myself and wrote a bunch of posts in early July, which filled the whole of summer schedule very well.

I had a nice summer, by the way, and I am about to begin my new occupation (PhD) which I am sure will take a lot of my free time. I know I tend to be very optimist (or I used to) and I would tell myself: "I can keep my schedule, at the end it is just one post a week".
But, even if it does not seem like it, I noticed that it takes a lot of time to write a good, well documented and effective (targeting the right audience and being coherent in difficulty) Linux post. And, on a side note, I consider myself a perfectionist and I generally like to go deep into stuff, so that does not help when I try to cut back on the time.

I decided to be more realistic this time, and I am apologizing in advance if I will not respect my weekly schedule. I will keep sure to write at least every fortnight, also because I would miss it otherwise. I also have another blog to maintain (see in contacts), so I am trying to split the time spent on them equally.

I have a treat for you, now. Open up that terminal of yours and run this string:

echo -e "\e[1;40m" ; clear ; while :; do echo $LINES $COLUMNS $(( $RANDOM % $COLUMNS)) $(( $RANDOM % 72 )) ;sleep 0.05; done|gawk '{ letters="abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789@#$%^&*()"; c=$4; letter=substr(letters,c,1);a[$3]=0;for (x in a) {o=a[x];a[x]=a[x]+1; printf "\033[%s;%sH\033[2;32m%s",o,x,letter; printf "\033[%s;%sH\033[1;37m%s\033[0;0H",a[x],x,letter;if (a[x] >= $1) { a[x]=0; } }}'

Cool uh? If it does not work, make sure you have gawk installed.
I will leave you with that and see you next time.

How to install software on Linux using repositories

This is a beginner guide on how to install software through terminal, using package managers. I will list the most common distributions (according to Distrowatch) and their command to search and install a certain software.

Search

Ubuntu, Debian, Mint (apt)

apt-cache search [software_name]

Fedora, Red Hat, CentOS (yum)

yum search [software_name]

Mandriva, Mageia (urpmi)

urpmi -y [software_name]

openSUSE (ZYpp)

zypper search [software_name]

Arch (pacman)

pacman -Ss [software_name]

Gentoo (portage)

emerge -s [software_name]

Install

Ubuntu, Debian, Mint (apt)

sudo apt-get install [software_name]

Fedora, Red Hat, CentOS (yum)

sudo yum install [software_name]

Mandriva, Mageia (urpmi)

urpmi [software_name]

openSUSE (ZYpp)

zypper install [software_name]

Arch (pacman)

pacman -S [software_name]

Gentoo (portage)

emerge [software_name]

Update 

Ubuntu, Debian, Mint (apt)

sudo apt-get upgrade

Fedora, Red Hat, CentOS (yum)

sudo yum update

Mandriva, Mageia (urpmi)

urpmi—auto-select

openSUSE (ZYpp)

zypper update

Arch (pacman)

pacman -Syu 

Gentoo (portage)

emerge -uD [software_name]

Uninstall

Ubuntu, Debian, Mint (apt)

sudo apt-get remove [software_name]

Fedora, Red Hat, CentOS (yum)

sudo yum remove [software_name]

Mandriva, Mageia (urpmi)

urpme [software_name]

openSUSE (ZYpp)

zypper remove [software_name]

Arch (pacman)

pacman -R [software_name]

Gentoo (portage)

emerge -C [software_name]

For Slackware, there is no package manager settling dependencies for you, and you can use pkgtool or slackpkg. Although, there is also slapt-get which somehow helps getting dependencies.


If you have a distribution that is based an older one, even if not listed here, it will most probably have the need the same commands to install software (e.g. Ubuntu is a Debian-based distribution, so they share the same installation commands).

How to batch rename files in Linux

Say you have a bunch of pictures and you want to rename them in a certain fashion. You can do it without the use of any external software and without painfully modifying the name of the pictures manually.

You can do it using just one command line! (even without using sed nor rename, if in a bash shell)

Strategy

Open a terminal and access the folder in which you have the files you want to modify:

$ cd ~/Desktop/Folder_name

Then use this command to modify the name of the files inside, to your need:

$ for file in *.* ; do mv [SOURCE_NAME] [DEST_NAME]; done

Examples

The examples below have increasing difficulty. Make sure to understand the previous ones before passing to the more complicated ones. Some knowledge of BASH will certainly help, but even you could use them blindly. Remember to be wise and make backups first, though.

If you have files with the same extension in the folder, such as JPG images, and you want to add a prefix, use this command:

$ for file in *.JPG ; do mv "${file}" "prefix_${file}"; done

If you want to add a suffix, instead, use this:

$ for file in *.JPG ; do mv "${file}" "${file%.JPG}_suffix.JPG"; done

And if your folder is full of mixed files, but you still want to add a suffix:

$ for file in *.*; do mv "${file}" "${file%.*}_suffix.${file#*.}"; done

If you want to number the files as well, sorting for modification time, use:

$ c=0; for file in $(ls -rt); do let c=${c}+1; mv "${file}" "prefix_${c}_suffix.${file#*.}"; done

Have fun renaming!

Monitor your badwidth usage: bwbar

The repositories include a nice little program to monitor your bandwidth usage. Very useful if you have a restriction on your bandwidth and you want to track your daily bandwidth usage.

The program is called bwbar. It generates a graphical readout of the bandwidth usage locally in a .png and a .txt file.

Installation

Ubuntu/Debian

$ sudo apt-get install bwbar

Fedora

$ sudo yum install bwbar

Usage

When installed, you need to modify the configuration file:

$ sudo emacs -nw /etc/default/bwbar

And uncomment the lines:

#DIR=/var/www
#RUNAS=www-data
#OPTIONS="eth0 1.5"

removing the # from the beginning of the line (it should change color from red when you do), and change the values depending on the directory you want the logs saved in, the user you want the program to run as, and other options you want to include when you run bwbar.


Then, you need to change the permissions, as following (if you kept the same folder):

$ sudo chown -R www-data:www-data /var/www/bwbar/

And then you can start the daemon by typing:

$ sudo /etc/init.d/bwbar start

You can see the bandwidth bar on http://[your-server-ip]/bwba.


If you want to track traffic, check out the package darkstat.

Mogrify: scale and crop multiple images in one command

When dealing with a lot of pictures, sometimes we want to make a common modification such as a change of resolution. In the case of multiple screenshots it is also common to crop all of the images at a fixed position (such as selecting only the same window).
Doing this job by hand, even just for three or four times, using the same parameters, is stressful and tiring, so we can ask for help to the terminal!

Mogrify is a command of the ImageMagick package and allows us to perform many operations on multiple images. In addition to scaling, resizing and cropping, mogrify can compress, convert, apply filter to images and much more.

Installation

For Fedora:

$ sudo yum install ImageMagick

For Ubuntu/Debian:

$ sudo apt-get install ImageMagick

Usage

The program overwrites the existing images, so, in order to prevent any unwanted modification, make a backup of the images or at least try the command on a sample image to make sure that it is the wanted modification.
Firstly, enter in the directory where the images that we want to modify are

$ cd folder_name

Then, say you want to scale the pictures to a resolution of 640x480:

$ mogrify -scale 640x480 *.* 

(note: if using the wildcard *.* , every single file in the folder will be modified by the command, so make sure that the folder contains only the files you want to modify or use different wildcards as *.jpg or e.g. *-2012.jpg)

If you want to scale the pictures keeping the aspect ratio, specify only the length in pixels and the height will be adjusted to keep constant the aspect ratio:

$ mogrify -scale 640 *.* 

Or using the same argument for the height instead of the width:

 $ mogrify -scale x480 *.* 

Or can also use a percentage of the original size, so if you want to halve the resolution:

$ mogrify -scale 50% *.*  

Or you can even modify only pictures bigger (or smaller) than the specified resolution and keep the others unchanged, respectively:

$ mogrify -scale '640x480>' *.*  

$ mogrify -scale '640x480<' *.*  

If you want to crop pictures in a window of 300x200 starting at pixels x=0 and y=100 where x is the offset in length from left to right and y is the offset in height from top to bottom, you need to use another option.
Since it is difficult to know the offset precise values a priori using gimp (a graphical editor) to take these values from the cursor position or selection can help. Then, use:

$ mogrify -crop 300x200+0+100 *.* 

If you want to compress images, use:

 $ mogrify -compress JPEG -quality 85 *.jpg 

where JPEG is the compression type and 85 is the quality for JPEG compression, which varies from 0 (lowest) to 100 (highest).


Or convert images from gif to jpeg format:

$ mogrify -format jpg -quality 85 *.gif

And rotate by an angle, e.g. 90 degrees:

$ mogrify -rotate 90 *.* 

Or if you want to rotate the image only in specific occasions, as if height is bigger than length (e.g. 480x640) (and viceversa), use, respectively:

$ mogrify -rotate '90>' *.*

$ mogrify -rotate '90<' *.*

Useful Links

Check out the official page of ImageMagick on Mogrify here, where you can find with more detail all the options of mogrify.

How to reinstall the network-manager if you accidentally removed it

Using the aptitude package manager in Debian, I noticed a message that suggested me to use the command:

$ apt-get autoremove

This would have removed the unused dependencies following to having removed some unused packages. The list was long and I skimmed quickly through it, noting the fact that there weren't important packages, as someone would expect.

Unfortunately, the Gnome network-manager was included in the list, and when I run the command it removed my possibility to connect to the internet and even to re-install the network-manager itself!

In this ironic situation, angry by being tricked to run a damaging command, I was already thinking to reinstall Debian. But wait! That would be a silly move.

The solution

In order to have an internet connection, you don't need the network manager. If you still have ethernet available, grab that cable and open a terminal:

$ sudo ifconfig eth0 up
$ sudo dhclient

This will make your ethernet working even without the Gnome network-manager. After this, you can run a:

$ sudo apt-get install network-manager-gnome

and have your wireless connection working again.

A suggestion

Do not use the auto-remove command if you do not know what it is removing. It might be a vital component for your system and damage it heavily.

You can use the auto-remove with the simulate option first (-s), or just deactivate the auto-remove in /etc/apt/apt.conf 

 // auto-remove breaks on meta packages
APT::Get::AutomaticRemove "0";
APT::Get::HideAutoRemove "1";

// Recommends are as of now still abused in many packages
APT::Install-Recommends "0";
APT::Install-Suggests "0";
Debug::pkgAutoRemove "0";

// PDiffs reduce the required download for apt-get update, but increase the
// CPU requirements and quite often fail.
Acquire::PDiffs "0";

// Remove apt unauthenticated warnings
APT::Get::AllowUnauthenticated "0";

Change language in Firefox and Thunderbird

Just after I installed Debian, I immediately felt the need to install Firefox and Thunderbird, as they are my favorite web and mail browsers.

As they are not present in the debian repositories, I had to use the ones from linuxmint (I added to my /etc/apt/sources.list):

deb http://debian.linuxmint.com/latest testing main contrib non-free

And after I installed Firefox and Thunderbird, I found myself with a Firefox in Afrikaans and Thunderbird in Arabic. Why this weird combination?

It is because of a bug in the Linux Mint repositories. When you go to install the browsers, they pick the first language pack they find, in alphabetical order!

In order to go back to english, you need to remove the language packs, using the package manager, and then install the english ones:

$ sudo apt-get remove firefox-l10n-af thunderbird-l10n-ar

This command alone actually automatically asked to install the English language packs. You can install also other languages by browsing in the repository:

$ apt-cache search firefox | grep l10n

For some reason, though, after this operations, the browsers did not work, so I tried:

$ sudo apt-get install firefox thunderbird

And everything worked like a charm, in a familiar language.

Ethernet connection not working in Debian after fresh installation

Just after I installed Debian Squeeze, at first boot, the wired ethernet connection, which worked so well during the install, was not working in the X system (Gnome).

I wondered a lot trying to find an error in ifconfig or dmesg, but could find none. After a while, I noticed that the "status" of the connection in Gnome whenever I clicked on the network manager icon was:

 "Wired device not managed"


The device was simply not activated in the interfaces!

In order to activate it, type from a terminal:

$ sudo emacs /etc/NetworkManager/NetworkManager.conf

(or 'vi' or 'nano' instead of emacs, depending on your favourite text editor)

Then change the line:

[ifupdown]

managed=false

to

[ifupdown]

managed=true

Then save the file and restart the network manager:

$ sudo service network-manager restart

And the wired connection should be working straight away.

Advertisements

I hate ads as much as you do, but for various reasons (already described here) I decided to put advertisements on this blog.

Thanks for understanding.

How to install non-free drivers during Debian Installation

As you might know from my previous post, I am a new Debian user. This is the first (and only, actually) "problem" I encountered installing Debian.

The Problem

I was installing on an amd64 architecture, from a live install on a bootable usb. The install required a network connection and I was using my wireless when I saw this notice:

Detect network hardware

Some of your hardware needs non-free firmware files to operate. The firmware can be loaded from removable media, such as a USB stick or floppy.

The missing firmware are: iwlwifi-5000-2.ucode iwlwifi-5000-1.ucode

If you have such media available now, insert it, and continue.

This error describes that the wireless card cannot be installed because some of the drivers used to install it are non-free (which means non-open source), and not present in the installer in order to keep it open-source. This is because my card is an Intel of 5000-series. What to do in this situation? There are two solutions. I would reccomend the first (like many other users), but if you want to play with your installation, give the second option a try.

The Solution

i. Use ethernet instead! You will still need an internet connection to install your Debian system from a live bootable usb/cd, but it is reccomended to use the cable during the installation. In this scenario, the wireless card will not be installed at first, but you will be able to install it later in an easier way, following the instructions: here. (ask in the comments if there are unclear steps)


ii. You will need to load the firmware files thorugh a media (usb or floppy). You will need to find the required files from a non-free repository and then put it in your usb drive (FAT filesystems preferred) in the main directory or a directory called /firmware, as described here. You can do this by switching workspace on the very same computer on which you are installing Debian.

Fedora -> Debian

After having used Fedora for the past 3 years, I finally decided to change distribution. And the decision pointed to Debian!

I wanted to move to a long-cycle release and more stable distribution and I found perfect ground in one of the oldest and most-established distributions based on the Linux kernel.

My following posts will naturally include more Debian-related problems, but I think that it is even better for my audience, as there are literally infinite sources of help for beginners for distributions such as Ubuntu and Fedora (their forums would be enough already).
In the case of Debian (which is known to be a distribution for the intermediate-expert users) there is still an extensive source of help, but it usually requires some deeper knowledge of Linux topics.

Despite I would not recommend Debian as first distro, I can assure that - being a beginner myself - you can do pretty well on Debian even if you are relatively new to Linux, and I will try to help, writing here in easy jargon about it!

Welcome, Debian (Squeeze)!

Everything you need to know about internet dongles on Linux

Yet another mobile broadband dongle is not working out of the box on your beloved Linux distribution.

If you are new to Linux, this might be one of the most annoying problems you will face, as there is a sea of different kinds of these internet dongles and they usually all require different drivers to be detected.

Fortunately, as almost every single wireless broadband dongle user seeks for help on Linux forums for his particular hardware, there is lot of help around from which you can guess and work out what is your problem.
But this makes the search messy, as often beginners get easily lost and reading discontinue posts on what to do is sometimes more difficult than trying to work out a solution on your own.

I am writing this post to settle down - hopefully once and for all - the main tests everybody (with any kind of mobile broadband dongle) should perform before asking for installation help in a forum. I've seen tens of posts about internet dongles which were solved with the same standard procedures. I will provide simple steps to interpret the problems and solutions to try to make your dongle working by yourself.

This guide is meant to be written for a generic (beginner) Linux user with a generic broadband dongle. All the commands should work on any Linux distribution. For your information, the distribution I used to make this tutorial is Debian Squeeze with Gnome (GUI), and the broadband modem is an obscure Olivetti Olicard 200, which was only known to be not very Linux-friendly.

1. Check that your system sees the hardware

Firstly, remove your PIN from the SIM card using a mobile phone, as it can unnecessarily complicate things.

Whenever you plug-in your usb broadband modem, the very first thing you always need to check is that your systems realizes that something has been plugged-in your usb port.

I am assuming that your mobile broadband dongle is working on the first place (can check on a Windows system, if you are not sure). A flashing led should show you that the electronics are working inside.

Run this command to see if the modem is recognized as plugged-in a usb port:

$ lsusb

If you see an error message, you most probably do not have installed the usbutils package. The output will be something similar to:

Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 003: ID 04b4:0060 Cypress Semiconductor Corp.
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 028: ID 0b3c:f000 Olivetti Techcenter
Bus 002 Device 003: ID 04f2:b044 Chicony Electronics Co., Ltd Acer CrystalEye Webcam
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

This is the list of the usb ports and what is connected to them, you should recognize your broadband dongle by the brand name. Sometimes the names do not coincide, so if you have troubles finding it in the list, disconnect it and run lsusb again to check which device is missing. The missing device is the broadband modem.

Take a note of the codes just before the brand name. In my case the numbers are:

0b3c:f000

where 0b3c is the vendor code and f000 is the product code.
This product code is used for storage usb dongles. As most of mobile broadband dongles are also capable to store data, the system is not recognizing the dongle as a modem. If you have this product code as well, proceed to Step 2.

If you see a code like XXXX or cXXX (where X is any number from 0-9), instead, your system already recognizes your dongle as a wireless broadband modem.
You can skip Step 2 and proceed to Step 3.

2. Switch device from usb to modem

(this step requires an internet connection, you can gather the packages from another pc if you are not able to connect on your own)

Your internet dongle is not recognized as such. We will use a software to let your system recognize it, called usb-modeswitch. You need to install two packages which are part of this program: usb-modeswitch and usb-modeswitch-data. Also install wdial and modemmanager, if for some reason are not installed already.

Once installed, you will need to type a single command to switch the mode of your internet dongle from storage usb-stick to modem.
Firstly, you will need some information:

- The vendor code, as found before usign lsusb (in my case 0b3c)
- The product code, as found before usign lsusb (in my case f000)
- A Hex key, associated to vendor and product code.

You already have vendor and product code, but in order to obtain the Hex key, you need to run the following command, which will retrieve the Hex key from your system.

 $ cat /etc/usb_modeswitch.d/[vendor code]\:[product code] | grep MessageContent

You need to substitute [vendor code] with your vendor code and [product code] with your product code. In my case, the command is:

 $ cat /etc/usb_modeswitch.d/0b3c\:f000 | grep MessageContent

This will print on your screen the Hex key. After you gathered this information, open the terminal and type:

$ sudo usb_modeswitch --default-vendor [vendor code] --default-product [product code] --message-content [hex key]

Which in my case, for a Olicard 200, is:

$ sudo usb_modeswitch --default-vendor 0x0b3c --default-product 0xf000 --message-content 5553424312345678c000000080010606f50402527000000000000000000000

 
After you run this code, you should see an output of this kind:

Looking for default devices ...
 Found devices in default mode, class or configuration (1)
Accessing device 003 on bus 001 ...
Getting the current device configuration ...
 OK, got current device configuration (1)
Using endpoints 0x01 (out) and 0x81 (in)
Using endpoints 0x01 (out) and 0x81 (in)
Inquiring device details; driver will be detached ...
Looking for active driver ...
 OK, driver found ("usb-storage")
 OK, driver "usb-storage" detached

SCSI inquiry data (for identification)
-------------------------
  Vendor String: USBModem
   Model String: MMC Storage   
Revision String: 2.31
-------------------------

USB description data (for identification)
-------------------------
Manufacturer: USBModem
     Product: HSPA Data Card
  Serial No.: 1234567890ABCDEF
-------------------------
Setting up communication with interface 0 ...
Using endpoint 0x01 for message sending ...
Trying to send message 1 to endpoint 0x01 ...
 OK, message successfully sent
Resetting response endpoint 0x81
Resetting message endpoint 0x01
 Device is gone, skipping any further commands
-> Run lsusb to note any changes. Bye.

If the command succeed, like in this case, run lsusb. When you do, you should note a change, as the internet dongle has now changed its product-code to "c005".

Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 002: ID 04b4:0060 Cypress Semiconductor Corp.
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 005: ID 0b3c:c005 Olivetti Techcenter
Bus 002 Device 003: ID 04f2:b044 Chicony Electronics Co., Ltd Acer CrystalEye Webcam
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Reboot and you can now proceed to the next step.

3. Install your internet dongle (the normal way)

If you are lucky enough, your broadband internet might be working already. In the GUI, click on the Network Manager icon and see if the Broadband connection is available. If it is, click on it to set up a new connection and proceed to Step 4.
Otherwise, try to run on a terminal:

$ sudo modprobe usbserial vendor=[vendor code] product=[product code]

Which for an Olicard 200 is:

$ sudo modprobe usbserial vendor=0x0b3c product=0xc005

This commands adds a loadable kernel module to recognize your device. Straight after you run it, your system should automatically detect your device and you can proceed with configuring your internet dongle.

If even this method did not work (as it did not on my Olidata 200), and your Network Manager does not recognize the device, it means that you will need to find another way to let it work, so jump this section and go to Step 5.

Now that you have your working dongle, you need to do some final setups before configuring it. In order to get the modprobe command run at every boot, you need to append the previous command to /etc/rc.local, by running as root:

# sed -i '$ d' /etc/rc.local; echo 'modprobe usbserial vendor=0x0b3c product=0xc005' >> /etc/rc.local; echo 'exit 0' >> /etc/rc.local

As usual, replace 0b3c and c005 with your vendor code and product code.

 4. Connect to the internet

You now need to set up a new connection with your broadband dongle. If a connection wizard opens, you can follow the instructions on screen and it will automatically configure your dongle.

It may happen that your dongle does not like the network manager configuration. You can get around the problem using wvdial.

$ sudo gedit /etc/wvdial.conf

and copy the following text:

[Dialer Defaults]
Modem = /dev/ttyUSB0
ISDN = 0
Modem Type = Analog Modem
Baud = 460800
Init1 = ATX3
Init2 = AT&F Q0 V1 E0 S0=0 &C1 &D2 +FCLASS=0
Init3 = at+cgdcont=1,"IP","[APN]"
Phone = *99#
Dial Attempts = 5
Stupid Mode = on
Dial Command = ATDT
Idle Seconds = 7200
Ask Passwords = 0
Password = "[PW]"
Username = "[UN]"
Carrier Check = on
New PPPD = 1
Auto DNS = on

You should modify [APN], [PW], [UN] to your needs.
Then, you can connect using

$ sudo wvdial

Congratulations, your dongle should be working now!

5. Install your internet dongle (the alternative way)

If the "normal" way to install your dongle did not work, you should consider using a very neat utility, which usually works with picky internet dongles.
It is called Sakis 3G.

You just need to download it and extract it:

$ wget http://www.sakis3g.org/versions/latest/binary-free/sakis3g.gz
$ gunzip sakis3g.gz
$ chmod +x sakis3g

Before running it, you need to remove the modemmanager application which controls mobile broadband devices through the GUI Network Manager. This is because it may conflict with Sakis 3G.

$ sudo apt-get remove modemmanager

Then you can run the application:

$ ./sakis3g --interactive

And you can follow the screens to configure the dongle and connect to the internet:

- Connect with 3G
- USB Device
- Select USB Device (the name should be familiar)
- Select Interface (only one should work as modem)
- Select APN (should auto-detect)

If you see a screen informing you that the device connected: congratulations, you managed to make your internet wireless dongle work on a Linux OS!

Please comment to suggest other methods or if you have any problems.

Pdftk: How to merge two pdf documents

Pdfs are an easy way to share documents through platforms without worrying about compatibility.

While this is clearly an advantage, pdfs lack the ease of modification that can be found in other documents types as Word documents. Essentially, pdfs are easy to read, but difficult to modify.

Fortunately, in Linux, there are many tricks to make the job easier. There is a piece of software called pdftk which allows great manipulation of pdfs.

Installation

Using Fedora:

$ sudo yum install pdftk

Using Ubuntu:

$ sudo apt-get install pdftk

 A few tricks to warm up

Pdftk allows to merge or split pdfs, rotate pdf pages, apply watermarks, attaching files to the pdfs, bursting the pdf in single pages, compressing it and repairing it in some cases, if corrupted.

In this post I will focus on how to merge pdf documents, but there is an extensive documentation on how to perform many other operations. That can be found on the PDFLabs website linked on the bottom of the page, or in the man pages.
The syntax to merge pdf files is relatively easy:

$ pdftk [input pdfs] cat output [output file name]

But its power comes from the possibility to use handles or wildcards. I'll provide a few examples.

To merge two pdf files into a file called combined.pdf :

$ pdftk file1.pdf file2.pdf cat output combined.pdf

If you want to merge all pdf files in the current directory can use the  wildcard *:

$ pdftk *.pdf cat output combined.pdf

 If you want to merge only specific pages of a pdf file with other pages of another pdf files (or the same) handles come handy:

$ pdftk A=file1.pdf B=file2.pdf cat A1 A2-10 B2 A3-7odd output combined.pdf

This may seem a bit complicated but I will explain: the input files are now called as two variables A and B, which are the handles. The codes in yellow are the rules to merge the two files, and it is here that lies the power of pdftk. It merges the pages from each file in the order we provide them. In this case:

   A1 = the first page of file1.pdf
   A2-10 = the pages from 2 to 10 of file1.pdf
   B2 = the second page of file2.pdf
   A3-7odd = the odd pages in the range from 3 to 7 of file1.pdf  (i.e. 3,5,7)

The even qualifier can be also used instead of odd, to take even pages only.

Main problem and solution

I came across pdftk when I needed to solve this problem:

I had scanned a set of papers through a fast document scanner, but since they were written on both sides, I found myself with two big pdfs, the first one containing odd pages only and the second one, containing even pages only, in reverse order!
I wanted a single file with the pages in the correct order but I could not find an easy way using a single command even using pdftk.

*** Update: This command would do the job for pdftk versions higher than 1.44:

 pdftk file2.pdf cat end-1 output - | pdftk file1.pdf - shuffle output combined.pdf

 Thanks to Andrew Peace ***

After some thinking I surrendered to the power of bash!

I decided to create a small bash script to form the command I needed to merge and order automatically the pages in a single file. For those who have done some basic bash, will immediately recognize the logic behind the script. For 50 pages pdfs:

#!/bin/bash
echo -e ' pdftk A=AQT.pdf B=AQTb.pdf cat \c'

max=50
for loop1 in {1..50}
do
    for loop2 in {A..B}
    do
    n=0
    if [[ ${loop2} == "A" ]] ; then
        let "n=$loop1"
    elif [[ ${loop2} == "B" ]] ; then
        let "n=$max-$loop1+1"
    fi
    echo -e  ${loop2}${n} '\c'
    n=0
    done
done

echo -e ' output combined.pdf \c'

Or you can download the script here and run it by simply typing:

$ ./pdftk_merge.sh

from the folder in which you downloaded the file.

The script prints automatically to the screen the command that you need to type (or better, copy) to merge the two set of pdfs! In this case the result is

pdftk A=file1.pdf B=file2.pdf cat A1 B50 A2 B49 A3 B48 A4 B47 A5 B46 A6 B45 A7 B44 A8 B43 A9 B42 A10 B41 A11 B40 A12 B39 A13 B38 A14 B37 A15 B36 A16 B35 A17 B34 A18 B33 A19 B32 A20 B31 A21 B30 A22 B29 A23 B28 A24 B27 A25 B26 A26 B25 A27 B24 A28 B23 A29 B22 A30 B21 A31 B20 A32 B19 A33 B18 A34 B17 A35 B16 A36 B15 A37 B14 A38 B13 A39 B12 A40 B11 A41 B10 A42 B9 A43 B8 A44 B7 A45 B6 A46 B5 A47 B4 A48 B3 A49 B2 A50 B1  output combined.pdf

Which, if executed from terminal, will correctly merge the two files in the right page order.

Useful links

Pdftk is a Pdflabs product: http://www.pdflabs.com. Many more examples on how to use it can be found here.

10 Nautilus tips and tricks

Nautilus is the file manager for Gnome. Gnome is a GUI for many Linux distributions and it is my favourite so far.

A file manager is the program that manages and lets you access your files. In plain words, it is the window with folders and stuff in it. Knowing how to use it properly can let you save a lot of time (unless you want to use the terminal, which could be even faster), so here is a list of a few tips and tricks to improve your file managing experience!

You can open a new window from terminal (if you have Gnome) with the command:

nautilus &

1. Change default "Open with" program for an extension

A quite common problem is the one of changing the default program with which a file is opened. If you have a Windows background, you'll endlessly search in the "Open with" from the right-click menu, but you will not find the answer there. Fortunately, the solution is easy:

-   Right-click on file with interested extension
-   Click on "Properties" at the bottom of the menu
-   Go on the "Open with" tab
-   Select wanted application and click on "Set as default"

Now the application you selected will be used by default as main application to open the files with the same extension with a double-click.

2. Multiple Panes in one window

Sometimes we want to move files from one folder to another and it would be convenient to have both folders somehow open in the same window to ease the transfer. This is possible!
In a window, just press the F3 button and a new pane will appear (initially with the same folder showing to both frames). You can switch from a pane to another just clicking on the wanted frame.

3. Select items matching a pattern

If you want to select multiple files with a certain name only or, say, with the same extension, there is an easier way rather than doing that manually!

Press Ctrl+S in a window and a dialog will open asking for the pattern. You can use the power of wildcards in this window!

The wildcards you can use are * and ?. Their working is very simple. Whenever the find program sees a *, it substitutes any random string in it. Basically, if you type just: * . It means everything.
For the ? wildcard, instead, it will replace only one character. Then, typing ????? will match any string containing 5 letters or numbers, including the extension.

It should be now clear how powerful can it be using wildcards for selecting files. I will provide a few examples:

*.jpg

Will select every file with extension .jpg, i.e. every image in a folder.

*2012*.jpg

This, instead, will select every picture in the folder containing 2012 in its name (note that *2012.jpg will only look for pictures having 2012 at the end of the filename, and viceversa for 2012*.jpg).

*.*

This will select all files (while * would also select folders).

???.*

This will select any file that contains 3 characters, such as lol.jpg or asd.txt.

4. Move, Copy, Link

You can use the click-and-drag options also present in Windows under Nautilus:

-   If you click and drag files without pressing any additional key, it will chose what to do depending on the destination. It will move files if the destination is local (same hard disk) and copy if the destination is not on the same File System (external drive).
-   If you click and drag pressing the Ctrl key, it will copy the files in any case.
-   If you click and drag pressing the Shift key, it will move the files in any case.
-   If you click and drag pressing both the Ctrl and Shift key, it will create a (soft) link to the files.

5. Move windows from anywhere

For users with low resolution screen, it may happen that some windows have the title bar (to move it) outside the borders of the screen.
This is quite annoying and it may be even impossible to close it, as the X button is hidden.
In order to move the Window, though, you can keep pressed the Alt key and left-click anywhere inside the window to move it!

6. Resize windows from anywhere

Similar to the previous one, but press Alt and middle-click with your mouse to resize the window without having to hunt for the border!

7. Run common scripts with one click

Nautilus has a default script folder in which you can put your bash scripts and run them by just selecting them (e.g. a script to remove files older than a certain date, or a script to clean the name of the selected files... you can really use your imagination here).

This is extremely useful and you can either download bash scripts or make your own. You just need to right-click and then select the "Scripts" menu, then the script which you want to run. You may find the script folder empty, initially.

In order to add your scripts to the list, you need to put your scripts in the scripts folder for nautilus. You can open it clicking on "Open scripts folder" from the same "Scripts" menu, but it is usually located in ~/.gnome2/nautilus-scripts . 

8. Quickly check number and size of multiple files

If you need to quickly know how big is a collection of files, it will be necessary to select them and check the bottom right corner of the folder. The number of the files selected, together with their total size is displayed in a pop-up balloon.

9. Keyboard shortcuts

This is a list of shortcuts to quickly perform the underlined actions. For some of them (e.g. Rename) you need to highlight a file first.

Rename: F2
Create new folder: Shift+Ctrl+N

Cut: Ctrl+X
Copy: Ctrl+C
Paste: Ctrl+V
Select All: Ctrl+A
Invert Selection: Shift+Ctrl+I

Add to Bookmarks: Ctrl+D
Search for Files: Ctrl+F

Move to Parent folder: Alt+↑
Move to Selected folder: Alt+↓
Move back: Alt+←
Move forward: Alt+→

10. Gnome 3 window tricks

This tricks will work only with Gnome 3.

Expand window: you can double-click on the title-bar to expand a window in any Gnome, but with Gnome 3 you can drag the title bar and push the window to the top to automatically expand it.

Divide screen between two windows: drag the window from the title bar and push it to the left (or right) to expand the window to half the size of the screen.

Open the activities: without clicking, push your mouse to the top right corner. This will open the "Activities menu".

Problem with the avahi daemon during the boot

A small reminder that I'm still alive plus a problem that almost scared me today (technologically speaking).

My Fedora 11 did me a trick and after an "apparently normal" shutdown yesterday, the avahi daemon failed during the boot hanging it on the login page (background image missing and plenty of error messages about the fail installing the daemon).
There were no ways to log in since after entering the password the laptop would have got stuck in an infinite loading machine. The other terminals (Ctrl+Alt+F2...) worked well.

What I did was then load the live cd and search for help over the internet being prepared to backup and re-install Fedora.
There weren't almost any hint on the web on how to solve this problem, if not for some issues with wireless cards, but nothing similar to my problem that completely blocked me from opening my GUI. I was lost.

Looking at these problems, anyway, enlightened me on how effective is the log system in Linux and the common suggestion to solve problems was, first of all and always, looking at the log in /var/log/messages, that is a record of everything Linux does in background or not.
That's how, after few minutes of searching and reading, that I stumbled upon this line:

Nov 23 09:49:01 acer avahi-daemon[1290]: write(): No space left on device
Nov 23 09:49:01 acer avahi-daemon[1290]: Failed to create PID file: No space left on device

That's even too clear. What seemed a serious problem on some damaged part of my distro was, instead, a problem of carelessly leaving the hard-disk with few bytes not even sufficient to start Gnome.

Deleting some files to free some space through terminal or using a live cd was enough to bring Gnome fully working again!

This is just to prove that sometimes problems that seem too complicate, can be solved just looking at a description of what is going on in the machine, given in detail by the machine itself.
Such an elegant service must not be wasted googleing the problem without mercy but, in most of the cases, "problems" can be solved locally using logs, man pages and your brain.

Hope I've learned the lesson!


Edit: thread found on fedoraforum.org (that is an excellent support forum, anyway) about a similar issue.

Problem with Grsync and (partial) remedy

I seize the opportunity of a recent problem with Grsync to talk about it.

Grsync is an useful GUI for the command rsync.
This command is very powerful, since it synchronizes two folders that can be on two different machines, or the same machine. Usually, if on the same machine, rsync is used for backup purposes.
Grsync provides a lot of options to make synchronizations and backups and is really user friendly: it only needs source and destination folders and some options to be set.

The synchronization consists in making the destination folder equal to the source folder.
This is a very common task, since it is common that a folder on a usb stick is used on other PCs and then we want to copy this folder on the PC at home. Copying the entire folder every time will be a slow method, especially if we are handling with gigabytes of data! Copying the files one by one requires a lot of memory and patience. Grsync requires just one click to identify (there are different methods that I will not list, for now) the differences between the two folders and copy only the files different, saving time, memory (human in this case) and cpu.


The usage is very simple: source and destination paths in the relative space and a few options; I'll explain the main ones, but a mouse roll-over the option gives a sufficient explanation of the option:
- Preserve time/owner/permissions/group: the destination files will have respectively the same modification time, the same owner (be careful, this is only a super-user option... in doubt un-tick it), the same permissions on the files (read-write-execute) and the same group.
- Delete on destination: makes a perfect synchronization, since file deleted in the source folder will be deleted also in the destination folder. Essential to have a faithful copy of the source, but to un-tick for incremental backups.
- Verbose: every deleted and copied file, error or message in general will be reported.

The next step is installing it! In Fedora you can use yum from a terminal:

sudo yum install grsync

or download and compile it from the official website.


---


Now let's focus on the problem: Grsync is not working now.
Why?
A conflict (maybe) with the gtk+ update (you don't need to know what gtk+ is, but if you really want... here).
Apparently, the new version of gtk: gtk+2.0-2.16.6 is not letting work grsync 0.9.1. The symptoms are different:
- Grsync window that won't open, but will be silently running in the processes.
- Message error, if grsync is run from terminal:

(grsync:#####): Gtk-CRITICAL **: gtk_combo_box_append_text: assertion `GTK_IS_LIST_STORE (combo_box->priv->model)' failed

repeated for how many grsync sessions are defined.
The problem has been discussed and there isn't still solution as reported here.
I absolutely don't have the solutions for this problem, since my beginner abilities could only help someone to explain or discover new cool programs as in this post.
Anyway, I've got two remedies, until someone find a radical solution to the problem.


The first remedy is pretty lame... but necessary if you want to stick to the program: a downgrade.

If you installed grsync with yum, that's very easy to do:

yum downgrade grsync

and it will downgrade grsync to the older version (0.6.3-2) that is working with the gtk+ update.


The second remedy is much more cool: using directly the rsync command in a terminal.
Rsync has a very easy syntax:

rsync [options] [source] [destination]

and it features a huge number of options. Check out the man pages to know more about this splendid command.
To automatize backups, you can also create a script. This is a BASH example that I use (I'm a BASH beginner too :D):

#!/bin/bash
#
# Syncronize using rsync "src" folder on the PC (source) with the "dest" folder
# on the stick (destination).
#

echo "Starting..."
rsync -vrlpEogth --delete --progress /home/user/src /media/usbstick/dest
exit

I'll introduce BASH and scripts in another post if you're not familiar with that.


That's all for now.

See you soon!


*** UPDATE ***
Thanks to the developer Piero Orsoni, the new version of the program, Grsync 0.9.2, is now full-working.
The version will be soon available also in the repositories.

Welcome!

Welcome to grep Linux!

This blog is one of the two daughter websites of the original Pepe Blog (now closed). I was writing about Science and Technology randomly, but over time, I switched my OS to Linux Fedora and got really into it. That resulted in a copious amount of blog posts about Linux and my experience using it.

That was getting readers confused and for this reason I decided to split the original blog into two blogs, one treating Linux in full and the other (Bad Scientist) about Science and various geek stuff.

I will post about cool news, tips and tricks on Linux, and various problems (with solutions) which I encountered in the perilous journey of working under a Linux OS. This blog is meant to be for beginners which have already some knowledge of a UNIX OS workings.

Thank you for reading,
Giulio