Linux Tips

Tuesday, November 15, 2022

Caterpillar - The CodePipeLine that looks like a caterpillar

Caterpillar - The CodePipeLine that looks like a caterpillar : Mitesh Singh Jat, CRED

At CRED, CaterPilLar CodePipeLine is a continuous integration / continuous delivery (CI/CD) service, which is being used to model, visualize, and automate the steps required to release the software.
Mitesh will explain how it can quickly model and configure the different stages of a software release process, automates the steps required to release software changes continuously.
CaterPilLar has helped to achieve not only CI/CD as a Code, but also Infrastructure as a Code.

Presented at Large Scale Production Engineering - India LSPE-In

Saturday, November 9, 2013

Skipping Ad while playing video using Mplayer

To skip Advertisements/unwanted scenes, or to mute certain parts of a movie, the Edit Decision Lists (EDL) are very helpful.

Please read the following link for more details:-
http://www.mplayerhq.hu/DOCS/HTML/en/edl.html

Friday, November 8, 2013

Mounting Debian ISO files as Offline Software Repository

If you are not connected to internet, then still you can install packages from Debian Installer ISOs. For example, if Debian 7.0 (Wheezy) ISOs are in directory /media/FreeAgent GoFlex Drive/Softwares/Debian_7.0_Wheezy directory and we want to mount these iso in /mnt/Debian/ directory. The following command can be used:

$ sudo $(which mount_debian.pl) -d /mnt/Debian -g -r "/media/FreeAgent GoFlex Drive/Softwares/Debian_7.0_Wheezy"
Using iso(s) dir '/media/FreeAgent GoFlex Drive/Softwares/Debian_7.0_Wheezy'
Using mount point dir '/mnt/Debian'
...
Total 11 iso(s) mounted successfully.

Unmounting of the iso(s) can be achieved by the following command:

$ sudo $(which mount_debian.pl) -d /mnt/Debian -u -g -r
Running: umount "/mnt/Debian"/*.iso
Running: rmdir "/mnt/Debian"/*.iso
Running: cp "/etc/apt/sources.list.net" "/etc/apt/sources.list"
Running: apt-get update
...

Usage of the command:

$ sudo $(which mount_debian.pl)
/home/mitesh/Perl/mount_debian.pl: Please provide Dir where Debian installer iso(s) are present.
USAGE: /home/mitesh/Perl/mount_debian.pl [options] iso_dir
          -a apt_sources_dir        default  /etc/apt
          -d mount_base_dir         default  /mnt/Debian
          -g                        debug on 
          -r                        apt-get update 
          -u                        unmount 
          iso_dir                   Dir where Debian installer iso(s) are present.

PS: iso_dir is not required with -u command

The mount_debian.sh perl script is given as follows:

#!/usr/bin/perl -w
#===============================================================================
#
#         FILE:  mount_debian.pl
#
#        USAGE:  ./mount_debian.pl 
#
#  DESCRIPTION:  
#
#      OPTIONS:  ---
# REQUIREMENTS:  ---
#         BUGS:  ---
#        NOTES:  ---
#       AUTHOR:  Mitesh Singh Jat (mitesh)
#      COMPANY:  
#      VERSION:  1.0
#      CREATED:  11/07/2013 10:30:04 PM IST
#     REVISION:  ---
#===============================================================================

use strict;
use warnings;

use Getopt::Std;
my $apt_sources_dir = "/etc/apt";
my $mount_point_base = "/mnt/Debian";
my $umount = 0;
my $reload = 0;
my $debug = 0;

sub usage()
{
    print STDERR "USAGE: $0 [options] <iso_dir>\n";
    print STDERR "          -a <apt_sources_dir>    apt sources dir: default  $apt_sources_dir\n";
    print STDERR "          -d <mount_base_dir>     mount point dir: default  $mount_point_base\n";
    print STDERR "          -g                      debug on \n";
    print STDERR "          -r                      apt-get update \n";
    print STDERR "          -u                      unmount \n";
    print STDERR "          <iso_dir>               Dir where Debian installer iso(s) are present.\n";
}

sub run_cmd()
{
    my $cmd = $_[0];
    print "Running: $cmd\n" if $debug;
    system("$cmd");
    my $retval = $?; 
    if ($retval != 0)
    {   
        print STDERR "Error in running: $cmd\nExit code = $retval\n";
    }   
    return ($retval);
}

sub run_cmd_get()
{
    my $cmd = $_[0];
    print "Running: $cmd\n" if $debug;
    my @outs = `$cmd`;
    my $retval = $?;
    if ($retval < 0)
    {
        print STDERR "Error in running: $cmd\nExit code = $retval\n";
        exit($retval);
    }
    chomp(@outs);
    return (@outs);
}
sub take_backup() 
{
    my $file1 = $_[0];
    my $file2 = $_[1];
    my $cmd = "cp \"$file1\" \"$file2\"";
    my $retval = &run_cmd("$cmd");
    if ($retval != 0)
    {
        print STDERR "$0: Cannot copy \"$file1\" to \"$file2\"\n";
        exit(-1);
    }
    return 1;
}
sub reload_repo()
{
    my $file1 = $_[0];
    my $file2 = $_[1];

    my $cmd = "cp \"$file1\" \"$file2\"";
    my $retval = &run_cmd($cmd);
    exit(-1) if ($retval != 0);

    $cmd = "apt-get update";
    $retval = &run_cmd($cmd);
    exit(-1) if ($retval != 0);
}

my %opts;
getopt('a:d:', \%opts);

foreach my $opt (sort keys %opts)
{
    if (!defined($opts{$opt}))
    {
        print STDERR "$0: Requires value for option '$opt'\n";
        &usage();
        exit(-1);
    }
    else 
    {
        if ($opts{$opt} =~ m/^-/) 
        {
            print STDERR "$0: Requires value for option '$opt'\n";
            &usage();
            exit(-1);
        }
    }
}
$apt_sources_dir = $opts{"a"} if (defined($opts{"a"}));
$mount_point_base = $opts{"d"} if (defined($opts{"d"}));
$debug = 1 if (defined($opts{"g"}));
$reload = 1 if (defined($opts{"r"}));
$umount = 1 if (defined($opts{"u"}));

my $apt_dvd = "$apt_sources_dir/sources.list.dvd";
my $apt_net = "$apt_sources_dir/sources.list.net";
my $apt_orig = "$apt_sources_dir/sources.list";

my $retval = 0;
my $cmd;
$retval = `whoami`;
chomp($retval);
unless ($retval eq "root")
{
    print STDERR "$0: Please run this script as 'root' user.\n";
    &usage();
    exit(-1);
}

unless ($umount)
{

if (@ARGV != 1) 
{
    print STDERR "$0: Please provide Dir where Debian installer iso(s) are present.\n";
    &usage();
    exit(-1);
}
my $iso_dir = "$ARGV[0]";
## Check for iso dir presence
unless (-d "$iso_dir") 
{
    print STDERR "$0: Please make sure the iso(s) dir '$iso_dir' is present\n";
    exit(-1);
}
print "Using iso(s) dir '$iso_dir'\n" if ($debug);

## Check for destination dir
unless (-d "$mount_point_base") 
{
    $cmd = "mkdir -p \"$mount_point_base\"";
    $retval = &run_cmd("$cmd");
    if ($retval != 0) 
    {
        print STDERR "$0: Cannot create dir '$mount_point_base'\n";
        exit(-1);
    }
}
print "Using mount point dir '$mount_point_base'\n" if ($debug);

## Taking backup of ftp/http line sources.list
my $is_backed_up = 0;
if (-f "$apt_net") 
{
    ## Take backup only when ftp / http (internet)
    $cmd = "grep -v -e \"^#\" -e \"^\$\" \"$apt_orig\" | egrep -c \"(ftp|http)://\"";
    my @nets = &run_cmd_get($cmd);
    if ($nets[0] > 0) 
    {
        $is_backed_up = &take_backup("$apt_orig", "$apt_net");
    }
}
else
{
    $is_backed_up = &take_backup("$apt_orig", "$apt_net");
}

## Get loop devices
$cmd = "ls /dev/loop[0-9]*";
my @temps = &run_cmd_get($cmd);
my %loop_devices;
my $nloop_devices = @temps;
foreach my $temp (@temps) 
{
    $loop_devices{$temp} = 1;
}
$cmd = "ls \"$iso_dir\" | grep -i \"\.iso\$\"";
@temps = &run_cmd_get($cmd);
my %iso_images;
my $niso_images = @temps;
foreach my $temp (@temps) 
{
    $iso_images{$temp} = 1;
}
$cmd = "df | awk '{print \$1}' | grep \"/dev/loop\"";
@temps = &run_cmd_get($cmd);
my %mounted_loop_devices;
my $nmounted_loop_devices = @temps;
foreach my $temp (@temps) 
{
    $mounted_loop_devices{$temp} = 1;
}

## Check for free loop devices
my $nfree_loops = $nloop_devices - $nmounted_loop_devices;
if ($niso_images > $nfree_loops)
{
    print "Less available loop devices $nfree_loops than isos $niso_images\n" if ($debug);
    my $req_loops = $niso_images - $nfree_loops; 
    if ($nmounted_loop_devices == 0) 
    {
        print "Reloading loop kernel module with extra $req_loops loop devices\n" if $debug;
        $cmd = "rmmod loop";
        $retval = &run_cmd($cmd);
        print "$retval\n";
        exit(-1) if ($retval != 0);
        $cmd = "modprobe loop max_loop=$niso_images";
        $retval = &run_cmd($cmd);
        exit(-1) if ($retval != 0);
    }
    else 
    {
        print STDERR "Please reload 'loop' kernel module. After unloading following loop devices...\n";
        foreach my $temp (sort keys %mounted_loop_devices)
        {
            print STDERR "$temp\n";
        }
        print STDERR "\nrmmod loop\n";
        my $temp = $nmounted_loop_devices + $req_loops;
        print STDERR "\nmodprobe loop max_loop=$temp\n\n";
        exit(-2);
    }
}

## Create entries in sources.list file
open(SFH, ">$apt_orig") or die("$0: Cannot write in file '$apt_orig'\n");
foreach my $iso (sort keys %iso_images)
{
    print "Mounting iso $iso ...\n" if $debug;
    my $dest_dir = "$mount_point_base/$iso";
    unless (-d "$dest_dir") 
    {
        $cmd = "mkdir -p \"$dest_dir\"";
        $retval = &run_cmd($cmd);
        exit(-1) if ($retval != 0);
    }
    $cmd = "df \"$dest_dir\" | grep -c \"^/dev/loop.*\.iso\$\"";
    my @mounts = &run_cmd_get($cmd); 
    if ($mounts[0] == 0) ## Mount only if not mounted
    {
        $cmd = "mount -o loop,ro \"$iso_dir/$iso\" \"$dest_dir\"";
        $retval = &run_cmd($cmd);
        exit(-1) if ($retval != 0);

        ## read distro version dir in dists/
        $cmd = "ls -l \"$dest_dir/dists\" | grep \"^d\" | awk '{print \$(NF)}' | head -1";
        my $version = `$cmd`;
        if (defined($version) and ($version ne ""))
        {
            $cmd = "ls -l \"$dest_dir/dists\"/*/ | grep \"^d\" | sort -u | awk '{print \$(NF)}' | awk '{sec=sec\" \"\$1} END{print sec}'";
            my $sections = `$cmd`;
            if (defined($sections))
            {
                chomp($sections);
                chomp($version);
                print SFH "deb file://$dest_dir $version $sections\n";
            }
        }
    }
}
print SFH "\n## Automatically generated by $0 ##\n";
close(SFH);
print "Total $niso_images iso(s) mounted successfully.\n" if $debug;

## Reload repository if sources.list is updated
if ($is_backed_up and $reload)
{
    &reload_repo("$apt_orig", "$apt_dvd");
}
exit(0);
}
#######################################
## umount code                       ##
#######################################

$cmd = "umount \"$mount_point_base\"/*.iso";
$retval = &run_cmd($cmd);
exit(-1) if ($retval != 0);
$cmd = "rmdir \"$mount_point_base\"/*.iso";
$retval = &run_cmd($cmd);
exit(-1) if ($retval != 0);
if ($reload)
{
    &reload_repo("$apt_net", "$apt_orig");
}
exit(0);

PS: The above script can be used with any Debian Package Management related distros like Ubuntu, Linux Mint, etc.

Wednesday, April 3, 2013

Pidgin: Keyboard becomes unresponsive after Status change

In Fedora 17 (i386), after changing status in pidgin 2.10.x, it is not possible to use keyboard in chat window. Keyboard becomes unresponsive. Even, I tried to compile and install newer pidgin 2.10.7, the issue remains same.

Then, after searching, I came across this bug https://developer.pidgin.im/ticket/15220 in Pidgin (actually GTK+), which suggests to

Right Click inside typing input box of Chat window > Input Methods > (Select any input method)

After this, input from keyboard is again possible.

Wednesday, March 27, 2013

Emacs in Daemon mode

When I started to use emacs as Java IDE, I have installed CEDET and JDEE. This make start time of emacs longer, even when I am not opening any Java file. Therefore, it is required to start Emacs in daemon mode. So that only one time startup is longer, and subsequent startup of emacs (emacs client to be precise) will be much faster.

I have written the following shell script (em, please place it in your $PATH), which starts the emacs daemon on first time only, and connects emacsclient to it.

#!/bin/bash

usr=$(whoami)
emacs_daemon="emacs --daemon"
cmd="ps auxww | grep \"$usr.*$emacs_daemon\" | grep -v grep"
#echo "$cmd"
is_running=$(eval "$cmd")
if [ -z "$is_running" ]
then
    echo "Starting emacs daemon"
    eval "$emacs_daemon"
    if [ $? -ne 0 ]
    then
        echo "$0: Cannot start $emacs_daemon" >&2
        exit
    fi
fi
emacsclient -a "" -t "$@"
exit $?

First Run:

$ em Hello.java
Starting emacs daemon

Warning: due to a long standing Gtk+ bug
http://bugzilla.gnome.org/show_bug.cgi?id=85715
Emacs might crash when run in daemon mode and the X11 connection is unexpectedly lost.
Using an Emacs configured with --with-x-toolkit=lucid does not have this problem.
("emacs")
Loading /usr/share/emacs/site-lisp/site-start.d/auto-complete-init.el (source)...
Loading /usr/share/emacs/site-lisp/site-start.d/auto-complete-init.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/emacs-color-theme-init.el (source)...
Loading /usr/share/emacs/site-lisp/site-start.d/emacs-color-theme-init.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/emacs-goodies-loaddefs.el (source)...
Loading /usr/share/emacs/site-lisp/site-start.d/emacs-goodies-loaddefs.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/focus-init.el (source)...
Loading /usr/share/emacs/site-lisp/site-start.d/focus-init.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/gnuplot-init.el (source)...
Loading /usr/share/emacs/site-lisp/site-start.d/gnuplot-init.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/gnus-bonus-init.el (source)...
Loading /usr/share/emacs/site-lisp/site-start.d/gnus-bonus-init.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/php-mode-init.el (source)...
Loading /usr/share/emacs/site-lisp/site-start.d/php-mode-init.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/rpm-spec-mode-init.el (source)...
Loading /usr/share/emacs/site-lisp/site-start.d/rpm-spec-mode-init.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/rpmdev-init.el (source)...
Loading /usr/share/emacs/site-lisp/site-start.d/rpmdev-init.el (source)...done
Loading /usr/share/emacs/site-lisp/cedet-1.1/common/cedet.el (source)...
Setting up CEDET packages...
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1
Setting up CEDET packages...done
Loading /usr/share/emacs/site-lisp/cedet-1.1/common/cedet.el (source)...done
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1                                                                                                  
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1                                                                                                  
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1
jde-java-font-lock: building names cache...
jde-java-font-lock: building names cache...empty
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1
Warning: cedet-called-interactively-p called with 0 arguments, but requires 1
Starting Emacs daemon.

Second and Subsequent Runs:

$ em Hello.java

Tuesday, February 19, 2013

Removing Empty Directories

We can remove empty directories inside a directory by using following command

$ ls -l * | grep -B1 "^total 0K" | grep -v -e "total 0K" -e "--" | cut -d: -f1 | xargs rmdir -v

The above command cannot delete recursively. Hence, using find command to delete recursively.

$ find . -type d -empty -delete





Posted by



mitesh.singh.jat




at

4:06 PM



No comments:
  


















Labels:
linux,
shell


        

          
        
Friday, November 16, 2012

          
        






Stopping Tumblerd in XFCE









Tumbler is a D-Bus service for applications to request thumbnails for various URI schemes and MIME types. It is an implementation of the thumbnail management D-Bus specification described here.




Issues with Tumblerd: 

Using more memory and CPU
Preventing removable storage devices from unmounting(then, ejecting)

External HDD
USB Pendrive
CD/DVD 

Usually, locking of external devices occur with Video files


$ lsof /run/media/mitesh/PHONE\ CARD
COMMAND   PID   USER   FD   TYPE DEVICE  SIZE/OFF NODE NAME
tumblerd 1268 mitesh    9r   REG   8,17  34902492 1625 /run/media/mitesh/PHONE CARD/video/abc.mp4
tumblerd 1268 mitesh   10r   REG   8,17  69869584 1624 /run/media/mitesh/PHONE CARD/video/def.mp4
tumblerd 1268 mitesh   11r   REG   8,17  35883924 1627 /run/media/mitesh/PHONE CARD/video/ghi.avi
tumblerd 1268 mitesh   14r   REG   8,17 293435929 1629 /run/media/mitesh/PHONE CARD/video/jkl/mno.flv
tumblerd 1268 mitesh   15r   REG   8,17 240015268 1630 /run/media/mitesh/PHONE CARD/video/jkl/pqr.flv



So, in order to get rid of this blocking process tumblerd, we can kill the tumblerd commands given by lsof, as:


 
$ lsof /run/media/mitesh/PHONE\ CARD/ | awk '{print $2}' | grep -v PID | xargs -L 1 kill -TERM



PS: Please replace /run/media/mitesh/PHONE\ CARD/ with the mount point of the removable device, which is not allowed to be umounted.







If we want to remove this tumblerd, we can give following command:


Fedora

$ sudo yum remove tumbler

Debian/Ubuntu

 $ sudo apt-get remove tumbler









Posted by



mitesh.singh.jat




at

11:02 AM



3 comments:
  


















Labels:
D-Bus,
Debian,
Fedora,
linux,
Ubuntu,
xfce










          
        

          
        
Friday, November 9, 2012

          
        






Encrypt Directory in Linux







Encrypt Directory

Preliminary Note
Installing eCryptfs
Setup

Setup (Simple)
Setup (Detail)

Encrypting Other Directory
Unmount





Encrypt Directory


This article describes basic usage of eCryptfs. It guides us through 
the process of creating a private and secure encrypted directory within 
our $HOME directory, where we can store all our sensitive files and 
private data.



In implementation eCryptfs differs from dm-crypt, which provides a 
block device encryption layer, while eCryptfs is an actual file-system 
– a stacked cryptographic file system to be exact.




References



eCryptfs on Debian squeeze


eCryptfs on Arch


eCryptfs on Gentoo




Preliminary Note

In this wiki, I will encrypt a directory /home/mitesh/Private, which is located in /home partition.
i.e. /home/mitesh/Private is an ordinary directory and does not use a partition of its own.


Installing eCryptfs


eCryptfs can be installed as following:



Debian



   $ sudo apt-get install ecryptfs-utils



Fedora



   $ sudo yum install ecryptfs-utils





Setup

Setup (Simple)

As a user, run the following command:

$ ecryptfs-setup-private


and follow the instructions.


Setup (Detail)



First, we create private directories.
$ mkdir /home/mitesh/Private
$ mkdir /home/mitesh/.Private
$ chmod 700 /home/mitesh/.Private
$ chmod 500 /home/mitesh/Private






Let's summarize



Actual encrypted data will be stored in ~/.Private directory (lower directory)


While mounted, decrypted data will be available in ~/Private directory (upper directory)



While not mounted nothing can be written to this directory


While mounted it has the same permissions as the lower directory 







eCryptfs can now be mounted on top of ~/Private. 
$ sudo mount -t ecryptfs /home/mitesh/.Private /home/mitesh/Private
Passphrase: <-- some_passphrase
Select cipher:
 1) aes: blocksize = 16; min keysize = 16; max keysize = 32 (not loaded)
 2) blowfish: blocksize = 16; min keysize = 16; max keysize = 56 (not loaded)
 3) des3_ede: blocksize = 8; min keysize = 24; max keysize = 24 (not loaded)
 4) twofish: blocksize = 16; min keysize = 16; max keysize = 32 (not loaded)
 5) cast6: blocksize = 16; min keysize = 16; max keysize = 32 (not loaded)
 6) cast5: blocksize = 8; min keysize = 5; max keysize = 16 (not loaded)
Selection [aes]: <-- ENTER
Select key bytes:
 1) 16
 2) 32
 3) 24
Selection [16]: <-- ENTER
Enable plaintext passthrough (y/n) [n]: <-- ENTER
Enable filename encryption (y/n) [n]: <-- ENTER
Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=bd28c38da9fc938b
WARNING: Based on the contents of [/root/.ecryptfs/sig-cache.txt],
it looks like you have never mounted with this key
before. This could mean that you have typed your
passphrase wrong.

Would you like to proceed with the mount (yes/no)? : <-- yes
Would you like to append sig [bd28c38da9fc938b] to
[/root/.ecryptfs/sig-cache.txt]
in order to avoid this warning in the future (yes/no)? : <-- yes
Successfully appended new sig to user sig cache file
Mounted eCryptfs 






Remember to take backup of /root/.ecryptfs/sig-cache.txt file, if /root partition gets reformatted
$ sudo cp -r /root/.ecryptfs /home/mitesh/






Verifying mounting of eCryptfs
$ mount | grep -i private
/home/mitesh/.Private on /home/mitesh/Private type ecryptfs (rw,relatime,ecryptfs_sig=e8d08163d274c68f,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_unlink_sigs)






Verifying encryption
$ cat ~/Private/test.txt
Some secret text
is present in this file.
$ sudo umount /home/mitesh/Private
$ cat ~/Private/test.txt
cat: /home/mitesh/Private/test.txt: No such file or directory
$ cat ~/.Private/test.txt
-#@%!*^(junk characters)^*!%@#-






Mounting without interaction (so passphrase must be in some removable device like USB pendrive)
$ sudo mount -t ecryptfs -o key=passphrase:passphrase_passwd_file=/mnt/usb/.ecryptfs-pass,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_passthrough=n,ecryptfs_enable_filename_crypto=n /home/mitesh/.Private /home/mitesh/Private
$ cat /mnt/usb/.ecryptfs-pass
passphrase_passwd=ThisIsAWeakPassword




Encrypting Other Directory


We can encrypt multiple directories, by putting those directories into /home/mitesh/Private and creating soft-links.
$ mv ~/Documents ~/Private
$ ln -s ~/Private/Documents ~/Documents



By following the above 2 steps, we have encrypted directory ~/Documents .



Unmount
$ sudo umount /home/mitesh/Private









Posted by



mitesh.singh.jat




at

11:01 AM



No comments:
  


















Labels:
Debian,
encrypt,
Fedora,
linux,
Ubuntu










          
        

          
        
Tuesday, August 21, 2012

          
        






Auto Mounting of Debian Offline Repository ISOs









If you are not connected to internet, then still you can install packages
from Debian Installer ISOs. 


For example, if Debian 6.0 (Squeeze) ISOs are in directory /media/FreeAgent GoFlex Drive/Softwares/Debian_6_0/
 directory and we want to mount these iso in /mnt/Debian/ directory. The following command can be used:
$ sudo $(which mount_debian.sh) /media/FreeAgent\ GoFlex\ Drive/Softwares/Debian_6_0 /mnt/Debian

The shell script mount_debian.sh is given below. 

You need to reload the debian repository (if internet repository is used before this command), to take packages from these mounted ISOs. Because the 
/etc/apt/sources.list is backed up and updated with these mounted ISOs.


$ sudo apt-get update 




In order to, unmount the ISOs, the following command can be given:
$ sudo $(which umount_debian.sh) /mnt/Debian/

This command will recover original /etc/apt/sources.list


mount_debian.sh
#!/bin/bash

DEBIAN_DIR="/mnt/Debian"
ISO_DIR="/mnt/temp/Softwares/Debian_5_0_5"
APT_DIR="/etc/apt"

## Run as root always
user_id=`whoami`

if [[ "$user_id" != "root" ]]
then
    echo "ERROR: please run this script as 'root' user."
    exit
fi    

if [ -d "$APT_DIR" ]
then
    echo -n ""
else
    echo "ERROR: This is not Debian System" 
    echo "       apt sources dir $APT_DIR is not present"
    exit
fi    

if [ $# -ge 1 ]
then
    ISO_DIR="$1"
else    
    echo "USAGE: $0 <ISO_DIR> <DEBIAN_DIR>"
    exit
fi    
    
if [ $# -ge 2 ]
then
    DEBIAN_DIR="$2"
fi    

## Check for ISO_DIR
if [ -d "$ISO_DIR" ]
then
    echo -n ""
else
    echo "$0: Please have dir with iso images: $ISO_DIR"
    exit
fi

if [ -d "$DEBIAN_DIR" ]
then
    echo -n ""
else
    mkdir -p "$DEBIAN_DIR"
fi
niso=0
nmount=0
apt_dvd="$APT_DIR/sources.list.dvd"
apt_net="$APT_DIR/sources.list.net"
apt_orig="$APT_DIR/sources.list"
if [ -f "$apt_net" ]
then
    echo -n ""
else
    # Take backup 
    cp $apt_orig $apt_net
fi    
echo "# Automatically generated by $0" > $apt_dvd
for iso in `ls "$ISO_DIR" | grep -i "\.iso$"`
do
    niso=`expr $niso + 1`
    n=`echo "$iso" | sed 's/.*\([0-9]\).iso/\1/'`
    echo "Mounting Disk $n..."
    dest_dir="$DEBIAN_DIR/Disk$n"
    if [ -d "$dest_dir" ]
    then
        echo -n ""
    else
        mkdir -p "$dest_dir"
    fi    
    echo "mount -o loop \"$ISO_DIR/$iso\" $dest_dir"
    mount -o loop "$ISO_DIR/$iso" $dest_dir
    if [ $? -ne 0 ]
    then
        echo "ERROR: Cannot mount $iso on $dest_dir";
    else
        nmount=`expr $nmount + 1`
        version=`ls -l $dest_dir/dists | grep "^d" | awk '{print $(NF)}'`
        secs=""
        for section in `ls -l $dest_dir/dists/stable/ | grep "^d" | awk '{print $(NF)}'`
        do
            secs=`echo "$secs $section"`
        done    
        echo "deb file://$dest_dir $version $secs" >> $apt_dvd
    fi    
done
echo ""
echo "Total Disks Mounted = $nmount/$niso"

echo ""
echo "Creating new $apt_orig..."
if [ $nmount -eq $niso ] 
then
    if [ $nmount -ne 0 ]
    then
        cp $apt_dvd $apt_orig
        echo "Created new $apt_orig"
    fi
fi    

exit 0



umount_debian.sh
#!/bin/bash

DEBIAN_DIR="/mnt/Debian"
ISO_DIR="/mnt/temp/Softwares/Debian_5_0_5"
APT_DIR="/etc/apt"

## Run as root always
user_id=`whoami`

if [[ "$user_id" != "root" ]]
then
    echo "$0: please run this script as root user."
    exit
fi    

if [ -d "$APT_DIR" ]
then
    echo -n ""
else
    echo "ERROR: This is not Debian System"
    echo "       apt sources dir $APT_DIR is not present"
    exit
fi    

if [ $# -ge 1 ]
then
    DEBIAN_DIR="$1"
else    
    echo "USAGE: $0 <DEBIAN_DIR>"
    exit
fi    
    
if [ -d "$DEBIAN_DIR" ]
then
    echo -n ""
else
    echo "$0: Debian dir $DEBIAN_DIR is not present"
    exit
fi

apt_dvd="$APT_DIR/sources.list.dvd"
apt_net="$APT_DIR/sources.list.net"
apt_orig="$APT_DIR/sources.list"
if [ -f "$apt_net" ]
then
    echo -n ""
else
    # Take backup 
    cp $apt_orig $apt_net
fi    
niso=0
nmount=0
for iso in `ls $DEBIAN_DIR | grep "^Disk[0-9]"`
do
    niso=`expr $niso + 1`
    n=`echo "$iso" | sed 's/.*\([0-9]\)$/\1/'`
    echo "Unmounting Disk $n..."
    dest_dir="$DEBIAN_DIR/Disk$n"
    if [ -d "$dest_dir" ]
    then
        echo "umount $dest_dir"
        umount $dest_dir
        if [ $? -ne 0 ]
        then
            echo "ERROR: Cannot umount $dest_dir";
        else
            nmount=`expr $nmount + 1`
        fi    
    fi    
done
echo ""
echo "Total Disks Unmounted = $nmount/$niso"

echo ""
#echo "Restoring $apt_orig..."
if [ $nmount -eq $niso ] 
if [ $nmount -eq $niso ] 
then
    if [ $nmount -ne 0 ]
    then
        cp $apt_net $apt_orig
         #echo "Restored $apt_orig"
    fi
fi    

exit 0



Related blogs: 

Blog: Offline Package Repository in Debian


Blog: Offline Package Repository in Debian 








Posted by



mitesh.singh.jat




at

12:53 PM



1 comment:
  


















Labels:
Debian,
DVD,
iso,
linux,
repository,
sources.list










          
        

          
        
Wednesday, April 25, 2012

          
        






Encrypt Partitions with dm-crypt LUKS










Encrypt Partitions

dm-crypt with LUKS

Create encrypted partition
Attach
Detach
Using multiple passphrase for same partition

dm-crypt without LUKS





Encrypt Partitions with dm-crypt LUKS

These instructions use the Linux dm-crypt (device-mapper) facility available on the 2.6 kernel. 
In this example, lets encrypt the partition /dev/sdc1, it could be however any other partition or disk, 
or USB or a file based partition created with losetup. In this case we would use /dev/loop0. 
The device mapper uses labels to identify a partition. We use sdc1 in this example, but it could be any string.


dm-crypt with LUKS

LUKS with dm-crypt has better encryption and makes it possible to have "multiple passphrase" for the same 
partition or to change the password easily. 
To test if LUKS is available, simply type 


# cryptsetup --help


if nothing about LUKS shows up, use the instructions below 
Without LUKS. 
First create a partition if necessary: 


# fdisk /dev/sdc

Create encrypted partition
# dd if=/dev/urandom of=/dev/sdc1          # Optional. For paranoids only (takes days)
# cryptsetup -y luksFormat /dev/sdc1       # This destroys any data on sdc1
# cryptsetup luksOpen /dev/sdc1 sdc1
# mkfs.ext3 /dev/mapper/sdc1               # create ext3 file system
# mount -t ext3 /dev/mapper/sdc1 /mnt
# umount /mnt
# cryptsetup luksClose sdc1                # Detach the encrypted partition

Attach
# cryptsetup luksOpen /dev/sdc1 sdc1
# mount -t ext3 /dev/mapper/sdc1 /mnt

Detach
# umount /mnt
# cryptsetup luksClose sdc1


Using multiple passphrase for same partition

We can store another password in a file (say ~mitesh/temp/key_file.txt) and use it.

# cryptsetup luksAddKey /dev/sdc1 ~mitesh/temp/key_file.txt 
Enter any passphrase: 
# cryptsetup luksOpen --key-file  ~mitesh/temp/key_file.txt /dev/sdc1 sdc1



dm-crypt without LUKS
# cryptsetup -y create sdc1 /dev/sdc1      # or any other partition like /dev/loop0
# dmsetup ls                               # check it, will display: sdc1 (254, 0)
# mkfs.ext3 /dev/mapper/sdc1               # This is done only the first time!
# mount -t ext3 /dev/mapper/sdc1 /mnt
# umount /mnt/
# cryptsetup remove sdc1                   # Detach the encrypted partition


Do exactly the same (without the mkfs part!) to re-attach the partition. If the password is not correct, the mount command will fail. In this case simply remove the map sdc1 (cryptsetup remove sdc1) and create it again.






Posted by



mitesh.singh.jat




at

11:54 AM



No comments:
  


















Labels:
linux










          
        

          
        
Saturday, March 31, 2012

          
        






Command Line Dictionary : meaning of






If we have only console, and we want to quickly see meaning of some word. Then we can use following perl script to get dictionary meaning of a word.
It uses perl, curl and html2text.


The source code of meaning_of.pl is :


#!/usr/bin/perl -w

## Search the dict.org

use strict;
use warnings;

sub print_usage() 
{
    print STDERR "USAGE: $0 <query>\n";
}

if (@ARGV < 1)
{
    &print_usage();
    exit(-1);
}

my $query = "$ARGV[0]";
$query =~ s/ /\%20/g;

my $cmd = "curl -# -A \"Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)\" -d \"Form=Dict1&Query=${query}&Strategy=*&Database=english\" \"http://www.dict.org/bin/Dict\" | html2text 2>/dev/null";
#system($cmd);
#exit($?);

open(MOFH, "$cmd | ") or die("$0: cannot execute: $cmd\n");

my $skip_lines = 10; # skip first 10 lines
my $line;
while ($line = <MOFH>)
{
    --$skip_lines;
    next if ($skip_lines > 0);
    chomp($line);
    print "$line\n";
}
close(MOFH);

exit(0);

Sample run of above perl script :


$ meaning_of.pl happiness

######################################################################## 100.0%
===============================================================================
4 definitions found for happiness
From The_Collaborative_International_Dictionary_of_English_v.0.48:

  Happiness \Hap"pi*ness\, n. [From Happy.]
     1. Good luck; good fortune; prosperity.
        [1913 Webster]

              All happiness bechance to thee in Milan! --Shak.
        [1913 Webster]

     2. An agreeable feeling or condition of the soul arising from
        good fortune or propitious happening of any kind; the
        possession of those circumstances or that state of being
        which is attended with enjoyment; the state of being
        happy; contentment; joyful satisfaction; felicity;
        blessedness.
        [1913 Webster]

     3. Fortuitous elegance; unstudied grace; -- used especially
        of language.
        [1913 Webster]

              Some beauties yet no precepts can declare,
              For there's a happiness, as well as care. --Pope.

     Syn: Happiness, Felicity, Blessedness, Bliss.

     Usage: Happiness is generic, and is applied to almost every
            kind of enjoyment except that of the animal appetites;
            felicity is a more formal word, and is used more
            sparingly in the same general sense, but with elevated
            associations; blessedness is applied to the most
            refined enjoyment arising from the purest social,
            benevolent, and religious affections; bliss denotes
            still more exalted delight, and is applied more
            appropriately to the joy anticipated in heaven.
            [1913 Webster]

                  O happiness! our being's end and aim! --Pope.
            [1913 Webster]

                  Others in virtue place felicity,
                  But virtue joined with riches and long life;
                  In corporal pleasures he, and careless ease.
                                                    --Milton.
            [1913 Webster]

                  His overthrow heaped happiness upon him;
                  For then, and not till then, he felt himself,
                  And found the blessedness of being little.
                                                    --Shak.
            [1913 Webster]
From WordNet_(r)_3.0_(2006):

  happiness
      n 1: state of well-being characterized by emotions ranging from
           contentment to intense joy [syn: happiness, felicity]
           [ant: unhappiness]
      2: emotions experienced when in a state of well-being [ant:
         sadness, unhappiness]
From Moby_Thesaurus_II_by_Grady_Ward,_1.0:

  117 Moby Thesaurus words for "happiness":
     Easy Street, acceptance, affluence, appropriateness, beatification,
     beatitude, becomingness, bed of roses, bewitchment, blessedness,
     bliss, blissfulness, blitheness, blithesomeness, brightness, cheer,
     cheerfulness, cheeriness, cheery vein, civility, cloud nine,
     clover, comfort, composure, content, contentedness, contentment,
     decency, decorousness, decorum, delectation, delight, ease,
     easy circumstances, ecstasy, ecstatics, elation, enchantment,
     enjoyment, entire satisfaction, eupeptic mien, euphoria,
     exaltation, exhilaration, exuberance, felicity, fitness,
     fittingness, fleshpots, fulfillment, gaiety, geniality,
     genteelness, gentility, gladness, gladsomeness, glee, good cheer,
     gracious life, gracious living, heaven, high spirits, hopefulness,
     intoxication, jollity, joy, joyance, joyfulness, joyousness,
     jubilation, lap of luxury, life of ease, loaves and fishes, luxury,
     meetness, optimism, overhappiness, overjoyfulness, paradise,
     peace of mind, pleasantness, pleasure, properness, propriety,
     prosperity, prosperousness, radiance, rapture, ravishment,
     reconcilement, reconciliation, resignation, rightness,
     rosy expectation, sanguine humor, sanguineness, satisfaction,
     security, seemliness, seventh heaven, success, suitability,
     sunniness, sunshine, the affluent life, the good life,
     thriving condition, transport, unalloyed happiness,
     upward mobility, urbanity, velvet, weal, wealth, welfare,
     well-being, winsomeness


From The_Devil's_Dictionary_(1881-1906):

  HAPPINESS, n.  An agreeable sensation arising from contemplating the
  misery of another.

===============================================================================
      Questions or comments about this site? Contact webmaster@dict.org







Posted by



mitesh.singh.jat




at

1:38 AM



No comments:
  


















Labels:
curl,
linux,
Perl










          
        

          
        
Thursday, February 23, 2012

          
        






Mount/Unmount USB pendrive/USB Harddisk





We have seen sometimes automounting of external drives(USB Pendrive/USB harddisk) work and sometimes it simply does not work. The pain increase when there are multiple partitions in the external drive. To resolve this issue, I have written a perl script to mount/unmount multiple partitions. 



The following perl script is useful in following cases:

1. Mounting/Unmounting(-u flag for unmounting) partitions in external drives.

2. Mounting/Unmounting partitions of internal drives too (-d , to (mount to)/(unmount from) any specific location).

3. This script also recognizes single vfat(FAT32) partition(When filesystem was created across the entire device: please refer mkfs.vfat -I) pen-drive too.

4. Others (if you find anything else, please comment :) )




#!/usr/bin/perl -w
#===============================================================================
#
#         FILE:  mount_usb_disks.pl
#
#        USAGE:  ./mount_usb_disks.pl 
#
#  DESCRIPTION:  Mount USB disks or pendrive
#
#      OPTIONS:  ---
# REQUIREMENTS:  ---
#         BUGS:  ---
#        NOTES:  ---
#       AUTHOR:  Mitesh Singh Jat (mitesh), <mitesh[at]yahoo-inc[dot]com>
#      COMPANY:  Yahoo Inc, India
#      VERSION:  1.0
#      CREATED:  02/22/2012 05:37:54 PM IST
#     REVISION:  ---
#===============================================================================

use strict;
use warnings;

use Getopt::Std;
my $mount_point_base = "/mnt/usb_disks";
my $umount = 0;

sub usage()
{
    print STDERR "USAGE: $0 [options] \n";
    print STDERR "          -d <mount_base_dir>     mount point dir: default  $mount_point_base\n";
    print STDERR "          -u                      unmount \n";
}

sub run_cmd()
{
    my $cmd = $_[0];
    print "Running: $cmd\n";
    system("$cmd");
    my $retval = $?;
    if ($retval < 0)
    {
        print STDERR "Error in running: $cmd\n$retval\n";
    }
    return ($retval);
}

sub run_cmd_get()
{
    my $cmd = $_[0];
    print "Running: $cmd\n";
    my @outs = `$cmd`;
    my $retval = $?;
    if ($retval < 0)
    {
        print STDERR "Error in running: $cmd\n$retval\n";
        exit($retval);
    }
    chomp(@outs);
    return (@outs);
}

my %opts;
getopt('d:', \%opts);

foreach my $opt (sort keys %opts)
{
    if (!defined($opts{$opt}))
    {   
        print STDERR "$0: Requires value for option '$opt'\n";
        &usage();
        exit(-1);
    }   
}
if (defined($opts{"d"}))
{
    $mount_point_base = $opts{"d"};
}
if (defined($opts{"u"}))
{
    $umount = 1;
}

my $retval = 0;
my $cmd;
$retval = `whoami`;
chomp($retval);
unless ($retval eq "root")
{
    print STDERR "$0: Please run this script as 'root' user.\n";
    &usage();
    exit(-1);
}
unless ($umount)
{
    print "Mounting on $mount_point_base\n";
    unless (-d "$mount_point_base") 
    {
        $retval = &run_cmd("mkdir -p $mount_point_base");
        exit($retval) if ($retval < 0);
    }

    my @temps = &run_cmd_get("mount | awk '{print \$1}' | grep \"^/dev/\"");
    my %mounted_devs;
    foreach my $temp (@temps)
    {
        $mounted_devs{$temp} = 1;
        $temp =~ s/\/sd([a-z]).*/\/sd$1/;
        $mounted_devs{$temp}++;
    }
    ## Take care of swap
    @temps = &run_cmd_get("cat /proc/swaps | awk '{print \$1}' | grep \"^/dev/\"");
    foreach my $temp (@temps)
    {
        $mounted_devs{$temp} = 1;
        $temp =~ s/\/sd([a-z]).*/\/sd$1/;
        $mounted_devs{$temp}++;
    }
    my %unmounted_devs;
    my $total = 0;
    @temps = &run_cmd_get("ls /dev/sd*");
    foreach my $temp (@temps)
    {
        next if (defined($mounted_devs{$temp}));
        ## Take care of logical partition, which cannot be mounted
        my @outs = &run_cmd_get("fdisk -s $temp");
        $retval = $outs[0];
        next if ($retval <= 1);      
        $unmounted_devs{$temp} = 1;
        $temp =~ s/\/sd([a-z]).*/\/sd$1/;
        if (defined($unmounted_devs{$temp}))
        {
            delete($unmounted_devs{$temp});
            $total--;
        }
        $total++;
    }
    if ($total == 0)
    {
        print "$0: No partition to mount.\nexiting...\n";
        exit(0);
    }

    print "\n--------------------------------------------\n";
    print "Following partitions are not mounted...\n";
    print "--------------------------------------------\n";
    foreach my $temp (sort keys %unmounted_devs) 
    {
        print "$temp\n";
    }
    print "--------------------------------------------\n";

    ## Now we can mount these
    foreach my $temp (sort keys %unmounted_devs)
    {
        my $mount_dir = $temp;
        $mount_dir =~ s/^\/dev\//$mount_point_base\//;
        print "Mounting $temp -> $mount_dir\n";
        unless (-d "$mount_dir")
        {
            $retval = &run_cmd("mkdir -p $mount_dir");
            exit($retval) if ($retval < 0);
        }

        $cmd = "mount $temp $mount_dir";
        $retval = &run_cmd("$cmd");
        if ($retval < 0)
        {
            print STDERR "!!!Cannot mount $temp on $mount_dir\n";
        }
    }
}
else
{
    unless ($mount_point_base =~ m/^\//) 
    {
        $retval = `pwd`;
        chomp($retval);
        $mount_point_base = "$retval/$mount_point_base";
    }
    print "Unmounting from $mount_point_base\n";
    $cmd = "mount | awk -F \" on \" '{print \$2}' | awk -F \" type \" '{print \$1}' | grep \"$mount_point_base\"";
    my @temps = &run_cmd_get("$cmd");
    if (@temps == 0)
    {
        print "$0: No partition to unmount.\nexiting...\n";
        exit(0);
    }
    print "\n--------------------------------------------\n";
    print "Following directories are mounted...\n";
    print "--------------------------------------------\n";
    foreach my $temp (@temps)
    {
        print "$temp\n";
    }
    print "--------------------------------------------\n";

    foreach my $temp (@temps)
    {
        $cmd = "umount $temp";
        $retval = &run_cmd("$cmd");
        if ($retval < 0)
        {
            print STDERR "!!!Cannot umount $temp\n";
        }
        else
        {
            &run_cmd("rmdir $temp");       # delete dir if empty
        }
    }
}
exit(0);






Posted by



mitesh.singh.jat




at

7:12 PM



No comments:
  


















Labels:
linux,
mount,
Perl




Newer Posts


Older Posts

Home




Subscribe to:
Posts (Atom)

Tuesday, November 15, 2022

Saturday, November 9, 2013

Friday, November 8, 2013

Wednesday, April 3, 2013

Wednesday, March 27, 2013

Tuesday, February 19, 2013

Friday, November 16, 2012

Friday, November 9, 2012

Encrypt Directory

Preliminary Note

Installing eCryptfs

Setup

Setup (Simple)

Setup (Detail)

Encrypting Other Directory

Unmount

Tuesday, August 21, 2012

Wednesday, April 25, 2012

Encrypt Partitions with dm-crypt LUKS

dm-crypt with LUKS

Create encrypted partition

Attach

Detach

Using multiple passphrase for same partition

dm-crypt without LUKS

Saturday, March 31, 2012

$ meaning_of.pl happiness

Thursday, February 23, 2012

Blog Archive

About Me