Advertisements

I hate ads as much as you do, but for various reasons (already described here) I decided to put advertisements on this blog.

Thanks for understanding.

How to install non-free drivers during Debian Installation

As you might know from my previous post, I am a new Debian user. This is the first (and only, actually) "problem" I encountered installing Debian.

The Problem

I was installing on an amd64 architecture, from a live install on a bootable usb. The install required a network connection and I was using my wireless when I saw this notice:

Detect network hardware

Some of your hardware needs non-free firmware files to operate. The firmware can be loaded from removable media, such as a USB stick or floppy.

The missing firmware are: iwlwifi-5000-2.ucode iwlwifi-5000-1.ucode

If you have such media available now, insert it, and continue.

This error describes that the wireless card cannot be installed because some of the drivers used to install it are non-free (which means non-open source), and not present in the installer in order to keep it open-source. This is because my card is an Intel of 5000-series. What to do in this situation? There are two solutions. I would reccomend the first (like many other users), but if you want to play with your installation, give the second option a try.

The Solution

i. Use ethernet instead! You will still need an internet connection to install your Debian system from a live bootable usb/cd, but it is reccomended to use the cable during the installation. In this scenario, the wireless card will not be installed at first, but you will be able to install it later in an easier way, following the instructions: here. (ask in the comments if there are unclear steps)


ii. You will need to load the firmware files thorugh a media (usb or floppy). You will need to find the required files from a non-free repository and then put it in your usb drive (FAT filesystems preferred) in the main directory or a directory called /firmware, as described here. You can do this by switching workspace on the very same computer on which you are installing Debian.

Fedora -> Debian

After having used Fedora for the past 3 years, I finally decided to change distribution. And the decision pointed to Debian!

I wanted to move to a long-cycle release and more stable distribution and I found perfect ground in one of the oldest and most-established distributions based on the Linux kernel.

My following posts will naturally include more Debian-related problems, but I think that it is even better for my audience, as there are literally infinite sources of help for beginners for distributions such as Ubuntu and Fedora (their forums would be enough already).
In the case of Debian (which is known to be a distribution for the intermediate-expert users) there is still an extensive source of help, but it usually requires some deeper knowledge of Linux topics.

Despite I would not recommend Debian as first distro, I can assure that - being a beginner myself - you can do pretty well on Debian even if you are relatively new to Linux, and I will try to help, writing here in easy jargon about it!

Welcome, Debian (Squeeze)!

Everything you need to know about internet dongles on Linux

Yet another mobile broadband dongle is not working out of the box on your beloved Linux distribution.

If you are new to Linux, this might be one of the most annoying problems you will face, as there is a sea of different kinds of these internet dongles and they usually all require different drivers to be detected.

Fortunately, as almost every single wireless broadband dongle user seeks for help on Linux forums for his particular hardware, there is lot of help around from which you can guess and work out what is your problem.
But this makes the search messy, as often beginners get easily lost and reading discontinue posts on what to do is sometimes more difficult than trying to work out a solution on your own.

I am writing this post to settle down - hopefully once and for all - the main tests everybody (with any kind of mobile broadband dongle) should perform before asking for installation help in a forum. I've seen tens of posts about internet dongles which were solved with the same standard procedures. I will provide simple steps to interpret the problems and solutions to try to make your dongle working by yourself.

This guide is meant to be written for a generic (beginner) Linux user with a generic broadband dongle. All the commands should work on any Linux distribution. For your information, the distribution I used to make this tutorial is Debian Squeeze with Gnome (GUI), and the broadband modem is an obscure Olivetti Olicard 200, which was only known to be not very Linux-friendly.

1. Check that your system sees the hardware

Firstly, remove your PIN from the SIM card using a mobile phone, as it can unnecessarily complicate things.

Whenever you plug-in your usb broadband modem, the very first thing you always need to check is that your systems realizes that something has been plugged-in your usb port.

I am assuming that your mobile broadband dongle is working on the first place (can check on a Windows system, if you are not sure). A flashing led should show you that the electronics are working inside.

Run this command to see if the modem is recognized as plugged-in a usb port:

$ lsusb

If you see an error message, you most probably do not have installed the usbutils package. The output will be something similar to:

Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 003: ID 04b4:0060 Cypress Semiconductor Corp.
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 028: ID 0b3c:f000 Olivetti Techcenter
Bus 002 Device 003: ID 04f2:b044 Chicony Electronics Co., Ltd Acer CrystalEye Webcam
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

This is the list of the usb ports and what is connected to them, you should recognize your broadband dongle by the brand name. Sometimes the names do not coincide, so if you have troubles finding it in the list, disconnect it and run lsusb again to check which device is missing. The missing device is the broadband modem.

Take a note of the codes just before the brand name. In my case the numbers are:

0b3c:f000

where 0b3c is the vendor code and f000 is the product code.
This product code is used for storage usb dongles. As most of mobile broadband dongles are also capable to store data, the system is not recognizing the dongle as a modem. If you have this product code as well, proceed to Step 2.

If you see a code like XXXX or cXXX (where X is any number from 0-9), instead, your system already recognizes your dongle as a wireless broadband modem.
You can skip Step 2 and proceed to Step 3.

2. Switch device from usb to modem

(this step requires an internet connection, you can gather the packages from another pc if you are not able to connect on your own)

Your internet dongle is not recognized as such. We will use a software to let your system recognize it, called usb-modeswitch. You need to install two packages which are part of this program: usb-modeswitch and usb-modeswitch-data. Also install wdial and modemmanager, if for some reason are not installed already.

Once installed, you will need to type a single command to switch the mode of your internet dongle from storage usb-stick to modem.
Firstly, you will need some information:

- The vendor code, as found before usign lsusb (in my case 0b3c)
- The product code, as found before usign lsusb (in my case f000)
- A Hex key, associated to vendor and product code.

You already have vendor and product code, but in order to obtain the Hex key, you need to run the following command, which will retrieve the Hex key from your system.

 $ cat /etc/usb_modeswitch.d/[vendor code]\:[product code] | grep MessageContent

You need to substitute [vendor code] with your vendor code and [product code] with your product code. In my case, the command is:

 $ cat /etc/usb_modeswitch.d/0b3c\:f000 | grep MessageContent

This will print on your screen the Hex key. After you gathered this information, open the terminal and type:

$ sudo usb_modeswitch --default-vendor [vendor code] --default-product [product code] --message-content [hex key]

Which in my case, for a Olicard 200, is:

$ sudo usb_modeswitch --default-vendor 0x0b3c --default-product 0xf000 --message-content 5553424312345678c000000080010606f50402527000000000000000000000

 
After you run this code, you should see an output of this kind:

Looking for default devices ...
 Found devices in default mode, class or configuration (1)
Accessing device 003 on bus 001 ...
Getting the current device configuration ...
 OK, got current device configuration (1)
Using endpoints 0x01 (out) and 0x81 (in)
Using endpoints 0x01 (out) and 0x81 (in)
Inquiring device details; driver will be detached ...
Looking for active driver ...
 OK, driver found ("usb-storage")
 OK, driver "usb-storage" detached

SCSI inquiry data (for identification)
-------------------------
  Vendor String: USBModem
   Model String: MMC Storage   
Revision String: 2.31
-------------------------

USB description data (for identification)
-------------------------
Manufacturer: USBModem
     Product: HSPA Data Card
  Serial No.: 1234567890ABCDEF
-------------------------
Setting up communication with interface 0 ...
Using endpoint 0x01 for message sending ...
Trying to send message 1 to endpoint 0x01 ...
 OK, message successfully sent
Resetting response endpoint 0x81
Resetting message endpoint 0x01
 Device is gone, skipping any further commands
-> Run lsusb to note any changes. Bye.

If the command succeed, like in this case, run lsusb. When you do, you should note a change, as the internet dongle has now changed its product-code to "c005".

Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 002: ID 04b4:0060 Cypress Semiconductor Corp.
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 005: ID 0b3c:c005 Olivetti Techcenter
Bus 002 Device 003: ID 04f2:b044 Chicony Electronics Co., Ltd Acer CrystalEye Webcam
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Reboot and you can now proceed to the next step.

3. Install your internet dongle (the normal way)

If you are lucky enough, your broadband internet might be working already. In the GUI, click on the Network Manager icon and see if the Broadband connection is available. If it is, click on it to set up a new connection and proceed to Step 4.
Otherwise, try to run on a terminal:

$ sudo modprobe usbserial vendor=[vendor code] product=[product code]

Which for an Olicard 200 is:

$ sudo modprobe usbserial vendor=0x0b3c product=0xc005

This commands adds a loadable kernel module to recognize your device. Straight after you run it, your system should automatically detect your device and you can proceed with configuring your internet dongle.

If even this method did not work (as it did not on my Olidata 200), and your Network Manager does not recognize the device, it means that you will need to find another way to let it work, so jump this section and go to Step 5.

Now that you have your working dongle, you need to do some final setups before configuring it. In order to get the modprobe command run at every boot, you need to append the previous command to /etc/rc.local, by running as root:

# sed -i '$ d' /etc/rc.local; echo 'modprobe usbserial vendor=0x0b3c product=0xc005' >> /etc/rc.local; echo 'exit 0' >> /etc/rc.local

As usual, replace 0b3c and c005 with your vendor code and product code.

 4. Connect to the internet

You now need to set up a new connection with your broadband dongle. If a connection wizard opens, you can follow the instructions on screen and it will automatically configure your dongle.

It may happen that your dongle does not like the network manager configuration. You can get around the problem using wvdial.

$ sudo gedit /etc/wvdial.conf

and copy the following text:

[Dialer Defaults]
Modem = /dev/ttyUSB0
ISDN = 0
Modem Type = Analog Modem
Baud = 460800
Init1 = ATX3
Init2 = AT&F Q0 V1 E0 S0=0 &C1 &D2 +FCLASS=0
Init3 = at+cgdcont=1,"IP","[APN]"
Phone = *99#
Dial Attempts = 5
Stupid Mode = on
Dial Command = ATDT
Idle Seconds = 7200
Ask Passwords = 0
Password = "[PW]"
Username = "[UN]"
Carrier Check = on
New PPPD = 1
Auto DNS = on

You should modify [APN], [PW], [UN] to your needs.
Then, you can connect using

$ sudo wvdial

Congratulations, your dongle should be working now!

5. Install your internet dongle (the alternative way)

If the "normal" way to install your dongle did not work, you should consider using a very neat utility, which usually works with picky internet dongles.
It is called Sakis 3G.

You just need to download it and extract it:

$ wget http://www.sakis3g.org/versions/latest/binary-free/sakis3g.gz
$ gunzip sakis3g.gz
$ chmod +x sakis3g

Before running it, you need to remove the modemmanager application which controls mobile broadband devices through the GUI Network Manager. This is because it may conflict with Sakis 3G.

$ sudo apt-get remove modemmanager

Then you can run the application:

$ ./sakis3g --interactive

And you can follow the screens to configure the dongle and connect to the internet:

- Connect with 3G
- USB Device
- Select USB Device (the name should be familiar)
- Select Interface (only one should work as modem)
- Select APN (should auto-detect)

If you see a screen informing you that the device connected: congratulations, you managed to make your internet wireless dongle work on a Linux OS!

Please comment to suggest other methods or if you have any problems.

Pdftk: How to merge two pdf documents

Pdfs are an easy way to share documents through platforms without worrying about compatibility.

While this is clearly an advantage, pdfs lack the ease of modification that can be found in other documents types as Word documents. Essentially, pdfs are easy to read, but difficult to modify.

Fortunately, in Linux, there are many tricks to make the job easier. There is a piece of software called pdftk which allows great manipulation of pdfs.

Installation

Using Fedora:

$ sudo yum install pdftk

Using Ubuntu:

$ sudo apt-get install pdftk

 A few tricks to warm up

Pdftk allows to merge or split pdfs, rotate pdf pages, apply watermarks, attaching files to the pdfs, bursting the pdf in single pages, compressing it and repairing it in some cases, if corrupted.

In this post I will focus on how to merge pdf documents, but there is an extensive documentation on how to perform many other operations. That can be found on the PDFLabs website linked on the bottom of the page, or in the man pages.
The syntax to merge pdf files is relatively easy:

$ pdftk [input pdfs] cat output [output file name]

But its power comes from the possibility to use handles or wildcards. I'll provide a few examples.

To merge two pdf files into a file called combined.pdf :

$ pdftk file1.pdf file2.pdf cat output combined.pdf

If you want to merge all pdf files in the current directory can use the  wildcard *:

$ pdftk *.pdf cat output combined.pdf

 If you want to merge only specific pages of a pdf file with other pages of another pdf files (or the same) handles come handy:

$ pdftk A=file1.pdf B=file2.pdf cat A1 A2-10 B2 A3-7odd output combined.pdf

This may seem a bit complicated but I will explain: the input files are now called as two variables A and B, which are the handles. The codes in yellow are the rules to merge the two files, and it is here that lies the power of pdftk. It merges the pages from each file in the order we provide them. In this case:

   A1 = the first page of file1.pdf
   A2-10 = the pages from 2 to 10 of file1.pdf
   B2 = the second page of file2.pdf
   A3-7odd = the odd pages in the range from 3 to 7 of file1.pdf  (i.e. 3,5,7)

The even qualifier can be also used instead of odd, to take even pages only.

Main problem and solution

I came across pdftk when I needed to solve this problem:

I had scanned a set of papers through a fast document scanner, but since they were written on both sides, I found myself with two big pdfs, the first one containing odd pages only and the second one, containing even pages only, in reverse order!
I wanted a single file with the pages in the correct order but I could not find an easy way using a single command even using pdftk.

*** Update: This command would do the job for pdftk versions higher than 1.44:

 pdftk file2.pdf cat end-1 output - | pdftk file1.pdf - shuffle output combined.pdf

 Thanks to Andrew Peace ***

After some thinking I surrendered to the power of bash!

I decided to create a small bash script to form the command I needed to merge and order automatically the pages in a single file. For those who have done some basic bash, will immediately recognize the logic behind the script. For 50 pages pdfs:

#!/bin/bash
echo -e ' pdftk A=AQT.pdf B=AQTb.pdf cat \c'

max=50
for loop1 in {1..50}
do
    for loop2 in {A..B}
    do
    n=0
    if [[ ${loop2} == "A" ]] ; then
        let "n=$loop1"
    elif [[ ${loop2} == "B" ]] ; then
        let "n=$max-$loop1+1"
    fi
    echo -e  ${loop2}${n} '\c'
    n=0
    done
done

echo -e ' output combined.pdf \c'

Or you can download the script here and run it by simply typing:

$ ./pdftk_merge.sh

from the folder in which you downloaded the file.

The script prints automatically to the screen the command that you need to type (or better, copy) to merge the two set of pdfs! In this case the result is

pdftk A=file1.pdf B=file2.pdf cat A1 B50 A2 B49 A3 B48 A4 B47 A5 B46 A6 B45 A7 B44 A8 B43 A9 B42 A10 B41 A11 B40 A12 B39 A13 B38 A14 B37 A15 B36 A16 B35 A17 B34 A18 B33 A19 B32 A20 B31 A21 B30 A22 B29 A23 B28 A24 B27 A25 B26 A26 B25 A27 B24 A28 B23 A29 B22 A30 B21 A31 B20 A32 B19 A33 B18 A34 B17 A35 B16 A36 B15 A37 B14 A38 B13 A39 B12 A40 B11 A41 B10 A42 B9 A43 B8 A44 B7 A45 B6 A46 B5 A47 B4 A48 B3 A49 B2 A50 B1  output combined.pdf

Which, if executed from terminal, will correctly merge the two files in the right page order.

Useful links

Pdftk is a Pdflabs product: http://www.pdflabs.com. Many more examples on how to use it can be found here.