Catatan Engineer

- WorkingArea
|
|
—-> Tek_informasi

- sysbti
|
|
—-> pde

Linux files systems work great. Especially ext3 is a mature file system and very robust. With journaling turned the filesystems stays consistent also on an unexpected power loss or system crash (wich normally never happens on linux ) and you don’t need long disk checks to repair the file system in such a case. There is almost no disk fragmentation and a lot of more cool features … But the best linux feature is: if you remove a file or directory it’s gone It’s not moved to trash, its not marked as deleted, there are no annoying messageboxes, it just does what you say. I love that!
But from time to time also experienced users make mistakes and you delete something wrong. As Murphy’s law says: “if anything can go wrong it will” and so you will not have a copy of the deleted files and the backup will be outdated. So what to do now?
I want to explain the procedure by using the problem I had as example.

How the trouble came up for me

As usual I was working on my Gentoo Linux laptop. I just wanted to add a new folder with sources to my subversion repository. I wanted to add first the folder, then some files within the folder…

Adding a folder to svn is simple, sometimes too simple…
svn add uaconsoleclientsample
Now I saw that this command added all files recursively, not just the folder. That means a lot of temporary files where added too.
The good thing with svn is, everthing you change is just done on your local working copy and not on the server. So it’s not a problem until you commit your changes. So lets remove the wrong files again before committing the changes.

Removing the files…
svn remove uaconsoleclientsample
svn: Use –force to override this restriction
svn: ‘uaconsoleclientsample’ has local modifications
Ok, remove didn’t work, use –force.

svn –force remove uaconsoleclientsample
Bad idea!!! The “svn remove” command not only removes files from scheduled add operations like I assumed. It removes it also from your local working copy.

So now my uncommited files were lost. This was the time where ext3grep came into play.

Recover lost files with ext3grep

I knew every new file could overwrite the data of my deleted files. So quickly unmount the partition. (The great thing, linux users have multiple partitions. We can unmount /home and continue to work as root)

Unmounting /home
Close my X Session
Alt-F1 -> log in as root
/etc/init.d/xdm stop
umount /home Cannot umount, files still in access
lsof | grep home -> artsd is still accessing home
killall artsd
umount /home Succeeded.
OK, done. Nothing can delete any more data now. Now relax, breath through and go to another Desktop PC, start the browser and ask google for a solution. -> ext3grep Ok, lets try this out. Login into laptop from the desktop, that’s more comfortable, just copy paste URLs to the shell etc. ssh -l root laptop

Installing ext3grep…
wget http://ext3grep.googlecode.com/files/ext3grep-0.7.0.tar.gz
tar -xvzf ext3grep-0.7.0.tar.gz
cd ext3grep
./configure
make
cd src
./ext3grep –help
Done, ext3grep is built and working

Lets search for my deleted folder:My laptop has an LUKS encrypted hard disk, and I devided this encrypted partition using LVM. For that reason my home partition is not a physical one like /dev/hda, but /dev/mapper/vg-home. Replace that with your partition you are using.
lt_gergap src # ./ext3grep /dev/mapper/vg-home –search uaconsoleclientsample
Running ext3grep version 0.7.0
Number of groups: 240
Minimum / maximum journal block: 1547 / 35890
Loading journal descriptors… sorting… done
Journal transaction 1202732 wraps around, some data blocks might have been lost of this transaction.
Number of descriptors in journal: 30993; min / max sequence numbers: 1202700 / 1204860
Blocks containing “uaconsoleclientsample”: 240017 242352 242355 242356 (allocated) 242357
(allocated) 242358 (allocated) 242359 255079 (allocated) 336393 336518 336526 395434 395435
395457 (allocated) 737282 (allocated) 984250 1346129 1868670 (allocated) 1869273 (allocated)
1950436 3915933 3915935 4069411 4087953 4216611 4292193 4292196 4292275 4530219 4538370
4538371 4538372 4538376 4538378 4538382 4538385 4543743 4543750 4543752 4544514 4544517
4544528 4544539 4550683 4550707 4655509 4655533 4670417 4670423 4689385 4689746 4785120
5046823 6525842 (allocated) 7370457 7805912
Wow, there are a lot of blocks. Why are some allocated? Lets check them all.

lt_gergap src # ./ext3grep /dev/mapper/vg-home –ls –block 240017
Running ext3grep version 0.7.0
Number of groups: 240
Minimum / maximum journal block: 1547 / 35890
Loading journal descriptors… sorting… done
Journal transaction 1202732 wraps around, some data blocks might have been lost of this transaction.
Number of descriptors in journal: 30993; min / max sequence numbers: 1202700 / 1204860
Group: 7
Block 240017 is Unallocated.
Hmm, seems not to be a directory, at least it contains nothing. Try the next one …

This operation takes some time and there are a lot of blocks. I’m lazy and before I do something twice I write a little script to automate this procedure.

1 #!/bin/sh
2

7 done

This script does the same procedure for each block in the list, shows the output on console and writes it to the file output.txt using tee. This way you can recheck the result later also in vim.

Analyzing the file output.txt I have seen only one interesting block:

…
Running ext3grep version 0.7.0
Number of groups: 240
Minimum / maximum journal block: 1547 / 35890
Loading journal descriptors… sorting… done
Journal transaction 1202732 wraps around, some data blocks might have been lost of this transaction.
Number of descriptors in journal: 30993; min / max sequence numbers: 1202700 / 1204860
Group: 57

Block 1869273 is a directory. The block is Allocated

.– File type in dir_entry (r=regular file, d=directory, l=symlink)
| .– D: Deleted ; R: Reallocated
Indx Next | Inode | Deletion time Mode File name
==========+==========+—————-data-from-inode——+———–+=========
0 1 d 657336 drwxr-xr-x .
1 2 d 657111 drwxr-xr-x ..
2 3 d 657337 drwxr-xr-x .svn
3 4 d 657345 drwxr-xr-x qtestclient
4 5 d 657354 drwxr-xr-x uaserverc
5 6 d 657444 drwxr-xr-x performanceclient
6 7 d 657534 drwxr-xr-x uademoserver
7 8 d 657687 drwxr-xr-x comdaproxy
8 9 d 657696 drwxr-xr-x examples_bin
9 10 d 657903 drwxr-xr-x uaconsoleclient
10 11 d 672247 drwxr-xr-x uaconsoleclientcpp
11 12 d 672766 drwxr-xr-x demoserver
12 13 d 672784 drwxr-xr-x uaserver
13 14 d 657946 drwxr-xr-x ascserver
14 15 d 672981 drwxr-xr-x server_ads
15 16 d 657947 drwxr-xr-x uaexpert
16 18 d 657949 drwxr-xr-x uasimpleclient
17 18 d 675465 D 1215690762 Thu Jul 10 13:52:42 2008 drwxr-xr-x bin
18 end d 656495 drwxr-xr-x uaconsoleclientsample
…
Yes, there it is. Let’s check the contents of this folder.

lt_gergap src # ./ext3grep /dev/mapper/vg-home –ls –inode 656495
Running ext3grep version 0.7.0
Number of groups: 240
Minimum / maximum journal block: 1547 / 35890
Loading journal descriptors… sorting… done
Journal transaction 1202732 wraps around, some data blocks might have been lost of this transaction.
Number of descriptors in journal: 30993; min / max sequence numbers: 1202700 / 1204860
Inode is Allocated
Loading vg-home.ext3grep.stage2……………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………….
WARNING: Rejecting a dir_entry (block 5714387) because it contains legal but unlikely characters.
Use –ls –block 5714387 to examine this possible directory block.
If it looks like a directory to you, and ’24′
looks like a filename that might belong in that directory, then add
–accept=’24′ as commandline parameter AND remove both stage* files!
……………………………………………………………………………………………………………………………………………………………………. done
The first block of the directory is 1348508.
Inode 656495 is directory “kde4/work/opcua/applications/uaconsoleclientsample”.
Directory block 1348508:
.– File type in dir_entry (r=regular file, d=directory, l=symlink)
| .– D: Deleted ; R: Reallocated
Indx Next | Inode | Deletion time Mode File name
==========+==========+—————-data-from-inode——+———–+=========
0 1 d 656495 drwxr-xr-x .
1 11 d 657336 drwxr-xr-x ..
2 4 r 2328216 D 1215690762 Thu Jul 10 13:52:42 2008 rrw-r–r– Makefile
3 4 r 2263308 D 1215690762 Thu Jul 10 13:52:42 2008 rrw-r–r– main.cpp
4 5 r 2328213 D 1215690762 Thu Jul 10 13:52:42 2008 rrw-r–r– qdevelop-settings.db
5 11 r 2328221 D 1215690762 Thu Jul 10 13:52:42 2008 rrw-r–r– uaconsolesampleclient.pro
6 7 r 2328215 D 1215690762 Thu Jul 10 13:52:42 2008 rrw-r–r– main.o
8 9 d 675466 D 1215690762 Thu Jul 10 13:52:42 2008 drwxr-xr-x moc
9 11 r 2262157 D 1215690762 Thu Jul 10 13:52:42 2008 rrwxr-xr-x uaconsolesampleclient
10 11 d 675465 D 1215690762 Thu Jul 10 13:52:42 2008 drwxr-xr-x PKI
11 end d 656701 drwxr-xr-x .svn
12 end r 2263307 D 1215690762 Thu Jul 10 13:52:42 2008 rr–r–r– .main.cpp.swx
Yes, everything is there. Now just restore the files you want using the right inode.

lt_gergap src # ./ext3grep /dev/mapper/vg-home –restore-inode 2263308
Running ext3grep version 0.7.0
Number of groups: 240
Minimum / maximum journal block: 1547 / 35890
Loading journal descriptors… sorting… done
Journal transaction 1202732 wraps around, some data blocks might have been lost of this transaction.
Number of descriptors in journal: 30993; min / max sequence numbers: 1202700 / 1204860
Restoring inode.2263308
mv RESTORED_FILES/inode.2263308 /home/kde4/work/opcua/applications/uaconsoleclientsample/main.cpp
lt_gergap src # ./ext3grep /dev/mapper/vg-home –restore-inode 2328221
Running ext3grep version 0.7.0
Number of groups: 240
Minimum / maximum journal block: 1547 / 35890
Loading journal descriptors… sorting… done
Journal transaction 1202732 wraps around, some data blocks might have been lost of this transaction.
Number of descriptors in journal: 30993; min / max sequence numbers: 1202700 / 1204860
Restoring inode.2328221
mv RESTORED_FILES/inode.2328221 /home/kde4/work/opcua/applications/uaconsoleclientsample/uaconsolesampleclient.pro That’s all. ext3grep did a great job for me. Thanks to the developers of that tool.

Use svn the correct way
Adding just a folder to svn would have been easy. Just use the -N option to work non recursive way.

svn add -N uaconsoleclientsample

http://www.mobilefx.com/web/documents/Fastrack_M1206.pdf

Manufacturer     : Wavecom
Model             : unknown (MULTIBAND 900E 1800)
Firmware        : 652a09gg.Q2406A 1489876 060706 17:19
IMEI             : 351621000512260
SIM IMSI        : 510101825265652

# apt-get install rsync
# ssh-keygen -t rsa
# scp .ssh/id_rsa.pub root@remotebox1.server.com:.ssh/authorized_keys2
# scp .ssh/id_rsa.pub root@remotebox2.server.com:.ssh/authorized_keys2

#rsync -avz -e ssh /home/ root@192.168.11.254:/home/dms/rsync-files
#rsync -avz -e ssh /etc/ root@192.168.11.254:/home/dms/rsync-files
#rsync -avz -e ssh /var/lib/mysql root@192.168.11.254:/home/dms/rsync-files
—-

—-
Isikan ke crontab …

Q. I’d like to configure my Debian box to backup two remote servers using rsnapshot software. It should make incremental snapshots of local and remote filesystems for any number of machines on 2nd hard disk located at /disk1 ( /dev/sdb2).

A. rsnapshot is perfect open source solution for making backups on local system. It supports both remote and local systems. From the man page:
rsnapshot saves much more disk space than you might imagine. The amount of space required is roughly the size of one full backup, plus a copy of each additional file that is changed. rsnapshot makes extensive use of hard links, so if the file doesn’t change, the next snapshot is simply a hard link to the exact same file. The following instructions are compatible with both Debian and Ubuntu Linux.

Required software / setup on local backup system
rsnapshot
rsync
ssh client
2nd hard disk ( RAID array is suggested) – you can also use primary hard disk
Password less login configured using ssh keys
/disk1/backup – Backup directory
/disk1/backup/server1 – Backup directory for remote server called server1
/disk1/backup/server2 – Backup directory for remote server called server2
/disk1/backup/localhost – Backup directory for local server
Required software on remote server
OpenSSH sshd server
Password less login configured using ssh keys
Step #1: Install rsync and rsnapshot software
Use apt-get command, enter:
$ sudo apt-get install rsync rsnapshot

Step #2: Configure passwordless login / public key based login
Type the following command
# ssh-keygen -t rsa
# scp .ssh/id_rsa.pub root@remotebox1.server.com:.ssh/authorized_keys2
# scp .ssh/id_rsa.pub root@remotebox2.server.com:.ssh/authorized_keys2

See how to configure RSA / DSA SSH public key based authentication.

Step #3: Configure rsnapshot utility
The configuration file is located at /etc/rsnapshot.conf. The configuration file requires tabs between elements and all drectories require a trailing slash. Just open config file using a text editor such as vi or gedit:
# vi /etc/rsnapshot.conf

OR
$ sudo vi /etc/rsnapshot.conf

Set snapshots root directory:

snapshot_root /disk1/backup/
Note you must separate snapshot_root and /disk1/ by a [tab] key i.e. type snapshot_root hit [tab] key once and type /disk1/backup/. All snapshots will be stored under this root directory (/disk1/backup/).

Configure backup policy

You can make hourly, daily, weekly or monthly snapshots of local and remote systems. To make a snapshot every four hours (six times a day) and keep a second set, which are taken once a day, and stored for a seven days, enter:

interval hourly 6
interval daily 7
Feel free to adapt configuration as per your backup needs.

Specify local and remote backup directories

Find out comments that read as follows:

###############################
### BACKUP POINTS / SCRIPTS ###
###############################
You need to comment out / delete default backup directories. To make snapshots for /home/, /etc/, /webroot/ directories to /disk1/backup/localhost, enter:

backup /home/ localhost/
backup /etc/ localhost/
backup /webroot/ localhost/
To backup remote server1 /home/, /etc/, /var/spool/mail/, /webroot/ directories to /disk1/backup/server1, enter:

backup root@remotebox1.server.com:/home/ server1/
backup root@remotebox1.server.com:/etc/ server1/
backup root@remotebox1.server.com:/webroot/ server1/
backup root@remotebox1.server.com:/var/spool/mail/ server1/
backup root@remotebox2.server.com:/home/ server2/
Save and close the file.

Test your config file for errors

Type the following to test your configuration file for errors
# rsnapshot configtest

Output:

Syntax OK
You can also run rsnapshot in a test mode to display its action:
# rsnapshot -t hourly

Step #4: Run rsnapshot for first time
To run first time, enter:
# rsnapshot hourly

Step #5: Configure cron job
Edit /etc/cron.d/rsnapshot file to setup backup snapshot job. This is a sample cron file for rsnapshot. The values used correspond to the examples in /etc/rsnapshot.conf. There you can also set the backup points and many other things. To activate this cron file you have to uncomment the lines below.
Feel free to adapt it to your needs.

0 */4 * * * root /usr/bin/rsnapshot hourly
30 3 * * * root /usr/bin/rsnapshot daily
0 3 * * 1 root /usr/bin/rsnapshot weekly
30 2 1 * * root /usr/bin/rsnapshot monthly
See crontab related faq for more information about cronjob under UNIX / Linux.

How do I exclude files from backup?
rsnapshot allows you to set the include and exclude parameters, if enabled, simply get passed directly to rsync. If you have multiple include/exclude patterns, put each one on a separate line. For example,
exclude_file /etc/rsnapshot.server1.conf

Append exclude file list to /etc/rsnapshot.server1.conf:
# vi /etc/rsnapshot.server1.conf

Exclude files matching pattern from backup:

var/lib/php/session/*
/var/spool/mail/nobody/*
cache/wp-cache-*.????
/var/logs/apache/access.log.*
/var/logs/apache/error.log.*
/linux-kernel/*
/tmp/cache/*
/var/lib/mysql/mysql.sock*
/tmp/php.socket-*
/tmp/*socket*

Save and close the file.

How do I backup remote MySQL database?
You can backup default database directory /var/lib/mysql. However, you can backup remote or local MySQL database with the following script:

#!/bin/sh
NOW=$(date +”%d-%m-%Y”)
MUSER=”MySQL-UserNAME” #root
MPASS=”MySQL-SERVER-PASSWORD” # mypassword
MHOST=”MySQL-SERVER-IP-ADDRESS” # server1
MYSQL=”$(which mysql)”
MYSQLDUMP=”$(which mysqldump)”
GZIP=”$(which gzip)”

DBS=”$($MYSQL -u $MUSER -h $MHOST -p$MPASS -Bse ‘show databases’)”
for db in $DBS
do
FILE=mysql-$db.$NOW-$(date +”%T”).gz
$MYSQLDUMP -u $MUSER -h $MHOST -p$MPASS $db | $GZIP -9 > $FILE
done
Now add following line to /etc/rsnapshot.conf file:
backup_script /root/scripts/mysql.backup.sh server1/mysql/

How do I restore backup?
You can simply copy back file using regular scp / cp command.

INSERT INTO outbox(
updatedindb, insertintodb, sendingdatetime, “text”, destinationnumber,
coding, udh, “class”, textdecoded, id, multipart, relativevalidity,
senderid, sendingtimeout, deliveryreport, creatorid)

VALUES (now(), ’1970-01-01 00:00:00′, ’1970-01-01 00:00:00′, NULL, ’08155031882′,
‘Default_No_Compression’, NULL, -1, ‘Haloo Papa Dimas… saya test ya’, nextval(‘outbox_id_seq’::regclass), FALSE, -1,
NULL, ’1970-01-01 00:00:00′, ‘default’, ”);

php-mysql-postgres.pdf

Gambaran Aplikasi yang ingin di buat
1.Terdiri dari beberapa modem
2.Terdapat satu database besar yang merupakan pusat database
3.Setiap modem akan berhubungan dengan satu database perantara yang terhubung dengan database besar tadi.

Langkah-langkah yang dilakukan untuk eksperimen
Buat databasenya (isinya sesuai dengan yang disediakan gammu). smsdMaster untuk menyimpan semua data dari berbagai database pembantu. smsd0 yang akan terhubung dengan modem0. smsd1 yang akan terhubung dengan modem1 dan smsd2 yang akan terhubung dengan modem2.
Buat file log untuk masing-masing modem (pada eksperimen ini di /var/log/gammu/gammulog, gammulog1 dangammulog2.
Pasang tiga buah modem (saya pakai itegno 3000).
Edit file gammurc tambahkan [gammu1] dan [gammu2] sesuaikan port dan logfilenya.
[gammu]
port = /dev/ttyUSB0
;model = 6110
connection = at115200
;synchronizetime = yes
logfile = /var/log/gammu/gammulog
logformat = textall
;use_locking = yes
;gammuloc = locfile
;startinfo = yes
;gammucoding = utf8
;usephonedb = yes
[gammu1]
port = /dev/ttyUSB1
;model = 6110
connection = at115200
;synchronizetime = yes
logfile = /var/log/gammu/gammulog1
logformat = textall
;use_locking = yes
;gammuloc = locfile
;startinfo = yes
;gammucoding = utf8
;usephonedb = yes
[gammu2]
port = /dev/ttyUSB2
;model = 6110
connection = at115200
;synchronizetime = yes
logfile = /var/log/gammu/gammulog2
logformat = textall
;use_locking = yes
;gammuloc = locfile
;startinfo = yes
;gammucoding = utf8
;usephonedb = yes
5. Copy file smsdrc menjadi smsdrc1 dan smsdrc2 kemudian sesuaikan isiannya
pada smsdrc1
[gammu]
port = /dev/ttyUSB1
;model = 6110
connection = at115200
;synchronizetime = yes
logfile = /var/log/gammu/gammulog1
logformat = textall
;use_locking = yes
;gammuloc = locfile
;startinfo = yes
;gammucoding = utf8
;usephonedb = yes
…
[smsd]
PIN =
logfile = /var/log/gammu/smsdlog1
commtimeout = 1
sendtimeout = 10
#receivefrequency = 0
#checksecurity = 1
#resetfrequency = 0
deliveryreport = yes
#phoneid = MyPhone1
#deliveryreportdelay = 10
#runonreceive = /some/script
# — SETTINGS FOR –smsd MYSQL or –smsd PGSQL ——-
user = root
password =
#pc = localhost
database = smsd1
pada smsdrc2
[gammu]
port = /dev/ttyUSB2
;model = 6110
connection = at115200
;synchronizetime = yes
logfile = /var/log/gammu/gammulog1
logformat = textall
;use_locking = yes
;gammuloc = locfile
;startinfo = yes
;gammucoding = utf8
;usephonedb = yes
…
[smsd]
PIN =
logfile = /var/log/gammu/smsdlog2
commtimeout = 1
sendtimeout = 10
#receivefrequency = 0
#checksecurity = 1
#resetfrequency = 0
deliveryreport = yes
#phoneid = MyPhone2
#deliveryreportdelay = 10
#runonreceive = /some/script
# — SETTINGS FOR –smsd MYSQL or –smsd PGSQL ——-
user = root
password =
#pc = localhost
database = smsd2
6. test koneksi modem dengan perintah
# gammu –identify
# gammu 1 –identify
# gammu 2 –identify
kalau sudah benar maka masing-masing akan memberikan respon berupa keterangan mengenai modem diantaranya
Manufacturer : Wavecom
Model : unknown (MULTIBAND 900E 1800)
Firmware : 652a09gg.Q2406A 1489876 060706 17:19
IMEI : 351621000512350
SIM IMSI : 510101825265653
7. Jalankan perintah
#gammu –smsd MYSQL /etc/smsdrc
#gammu –smsd MYSQL /etc/smsdrc1
#gammu –smsd MYSQL /etc/smsdrc2
selanjutnya silahkan mencoba untuk mengirim dan menerima sms pada masing-masing modem. untuk mengirim dari modem 0 masukkan data pada tabel outbox database smsd, modem1 pada smsd1 dan modem2 pada smsd2.

apt-get update
apt-get install pgsql
Create Language
Example plpgsql

su postgres
createlang plpgsql template1
exit
Change authentication method
We need to edit file pg_hba.conf to change authentification method for accessing PostgreSQL database.

cp /etc/postgresql/pg_hba.conf /etc/postgresql/pg_hba.confbak
vi /etc/postgresql/pg_hba.conf
Find this section

# TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD
# Database administrative login by UNIX sockets
local all postgres ident sameuser
#
# All other connections by UNIX sockets
local all all ident sameuser
#
# All IPv4 connections from localhost
host all all 127.0.0.1 255.255.255.255 ident sameuser
#
# All IPv6 localhost connections
host all all ::1 ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff ident sameuser
host all all ::ffff:127.0.0.1/128 ident sameuser
#
# reject all other connection attempts
host all all 0.0.0.0 0.0.0.0 reject
Edit that section to

# TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD
# Database administrative login by UNIX sockets
local all postgres ident sameuser
#
# All other connections by UNIX sockets
local all all password
#
# All IPv4 connections from localhost
host all all 127.0.0.1 255.255.255.255 password
#
# All IPv6 localhost connections
host all all ::1 ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff password
host all all ::ffff:127.0.0.1/128 password
#
# reject all other connection attempts
host all all 0.0.0.0 0.0.0.0 reject
Restart PostgreSQL Server
/etc/init.d/postgresql restart
Create a New Database
Example wordpress

su postgres
createdb -T template1 wordpress
exit
Create a New User
Example: User supriyadisw with password cak3p

su postgres
createuser supriyadisw -P
Enter password for new user: cak3p [enter]
Enter it again: cak3p [enter]
Shall the new user be allowed to create databases? (y/n) y [enter]
Shall the new user be allowed to create more new users? (y/n) n [enter]
CREATE USER
exit
Login to PostgreSQL
pgsql -U supriyadisw wordpress [enter]
Password: cak3p [enter]
Good Luck