crontab

The software utility Cron is a time-based job scheduler in Unix-like computer operating systems. People who set up and maintain software environments use cron to schedule jobs (commands or shell scripts) to run periodically at fixed times, dates, or intervals. read more at WikiPedia

  • FaF (File Anomaly Finder) is a wrapper for the *nix 'find' utility. It generates audit reports for data matching specific characteristics; such data as setgid/setuid, unowned, and more. The objectives are simply to create a simple anomaly finder that identifies common flawed permissions or otherwise suspicious file system characteristics.

    The main features of FaF are:
    • simplistic and to the point audit reports
    • easy setup and configuration
    • audits emailed to customizable address or user
    • ideal for web servers or general purpose workstations
    • audits of setgid/setuid, hidden, unowned, & world writable data
    • very portable
     http://www.r-fx.org/faf.php # wget http://www.r-fx.ca/downloads/faf-current.tar.gz
    # tar xvf  faf-current.tar.gz

    # cd faf*
    # ./install.sh

    Install path:     /usr/local/faf/
    Config path:     /usr/local/faf/conf.faf
    Executable path: /usr/local/sbin/faf


    Why do you need such tool?
    Never trust anyone, including sometimes yourself ;-) this tool correctly used just insured You that You will never forget any files with too much permissions. It may also reveal a hacker, putting some new files under the user nobody...

    What to do with the output?

    You'll have to react differently for each occurrence in the report....

    SUID/SGID Binaries

    Sticky bit was used on executables in linux (which was used more often) so that they would remain in the memory more time after the initial execution, hoping they would be needed in the near future. But since today we have more sophisticated memory accessing techniques and the bottleneck related to primary memory is diminishing, the sticky bit is not used today for this. Instead, it is used on folders, to imply that a file or folder created inside a sticky bit-enabled folder could only be deleted by the creator itself. A nice implementation of sticky bit is the /tmp folder,where every user has write permission but only users who own a file can delete them. Remember files inside a folder which has write permission can be deleted even if the file doesn't have write permission. The sticky bit comes useful here.

    SUID or SetUID bit, the executable which has the SUID set runs with the ownership of the program owner. That is, if you own an executable, and another person issues the executable, then it runs with your permission and not his. The default is that a program runs with the ownership of the person executing the binary.

    Consider also reading:
    What are the SUID, SGID and the Sticky Bits?

    You can find them also manually by entering:
    # find / -type f \( -perm -04000 -o -perm -02000 \;
    The SGID bit is the same as of SUID, only the case is that it runs with the permission of the group. Another use is it can be set on folders,making nay files or folders created inside the SGID set folder to have a common group ownership.

    files in/srv  (http root folder)
       You should accept NO files with SUID/SGID in http root folder. Remove them all 
            # find /srv -type f \( -perm -04000 -o -perm -02000 \) -exec  chmod \;

    No Owner/Group
    May also be an indication an intruder has accessed your system...
    Can also be found manually by typing:
    # find / \( -nouser -o -nogroup \) -print
    files in/srv (http root folder)

    Permissions and ownership are linked together to make your server work peacefully. The basic idea is always to give the minimum rights to the file.

    A rule for thumbs would be:
    read only for all file, r--r--r-- or r---------
    read, execute for all directory r-xr-xr-xor r-x------
    The problem is that apache and PHP also run under their own user...

    A very informative article explaining the problem on a concrete example (Gallery2) can be found at  http://codex.gallery2.org/Gallery2:Security

    At least (worst),when apache run as wwwrun user in www group, in your HTTP directory
    # chown -R wwwrun .
    # chgrp  -R www .
    then all files has to be  rw- --- --- and directory r-x------
    Advantages:you can use Joomla! administrator panel
    BUT: any bug in PHP code, attack can read or overwrite any files! -> highly insecure

    Better would be for all files/dir in your HTTP directory to changes accordingly to the right web user!
    # chown -R cedric .
    # chgrp  -R psacln  .
    Change all files/directories that has to be written  by apache (cache directories) to
    # chown -R wwwrun cache
    # chgrp  -R www cache
    Advantages: a bug in apache/php, or attack can not touch any of your files.
    BUt: if PHP do not run under your user, the Joomla! panel wont be usable, as Apache/PHP wont be able to install any new components/images.

    Files in /must generally only be available to root
    # chown -R root /etc
    #chgrp  -R root /etc
    # find /etc -f -exec chmod 600 {} /;

    World Writable

    files in/srv
    must be avoid at any costs! This line remove the world writable bit to  all files in /srv
    # find /srv -f -exec chmod o-w {} /;
    This line remove the world writable bit to  all directories in /srv
    # find /srv -d -exec chmod o-w {} /;
    Files in /
    You should ignores /proc files, /dev files (hundreds of these are correctly world writable),
    Symbolic (soft) links (which should have mode 777), directories with the sticky (save text) bit on, and
    sockets, as that is relatively safe.
    Hidden Files/Paths

    You should normally have no such files! try to understand why (look in google), open them and/or move/delete them
  • backup_debian_ubuntu_server

    I am using ReoBack for this duty

    REOBack (pronounced "ray-o-back") is a backup solution designed for Linux users and system administrators. It is designed to be simple to set up, and easy to use. It is great as a quick solution for those who procrastinate about backups. It supports automatic full/incremental backups of files you define, remote backups via NFS or FTP, as well as auto deletion of old backups.

    And here is my how to install for Debian /Ubuntu distribution taken from my notes

    Download Reoback 1.0.3, convert it into a Debian package with alien and install it

    wget http://puzzle.dl.sourceforge.net/sourceforge/reoback/reoback-1.0-3.noarch.rpm
    apt-get install alien
    alien reoback-1.0-3.noarch.rpm
    dpkg -i reoback_1.0-4_all.deb

    Configure ReoBack by editing the file settings.conf

    vi /etc/reoback/settings.conf
    Here is a sample configuration file to save your backup to a remote FTP server (but it could be also a NFS share)
    host            = myhostname.com
    backupdays      = 7
    files           = /etc/reoback/files.conf
    tmpdir          = /var/lib/reoback/tmp/
    datadir          = /var/lib/reoback/data/
    localbackup     = /var/lib/reoback/backups/
    keeplocalcopy   = 1
    remotebackup    = 1
    rbackuptype     = FTP
    localmount      = /mnt/server/
    remotehost      = xxxxxxxxx
    remotepath      = /reobackup/
    ftpuser         = xxxxxxxxx
    ftppasswd       = xxxxxxxxx

    Specify which files on your server  need to be saved by editing the file files.conf

    vi /etc/reoback/files.conf

    Here is a sample, I exclude some files that are changing all the time since they are maintained by the Linux kernel or some processes

    File: homes
    /home/
    
    File: var
    /var
    Skip: /var/run/*
    Skip: /var/lib/mysql/*
    Skip: /var/lib/reoback/*
    
    File: mysql
    /var/lib/mysql
    /tmp/mysql.sock
    Skip: /var/lib/mysql/mysql.sock
    Skip: /var/lib/mysql/mysqld.pid
    
    File: plesk
    /opt/psa
    /etc/psa
    /usr/local/psa

    Adapt the location path of these 2 files (files.conf / settings.conf) in  run_reoback.sh

    vi /etc/reoback/run_reoback.sh

    content of file

    # Location of the configuration file.
    config="/etc/reoback/settings.conf"
    
    # Change to reflect where REOBack is installed
    reoback="/usr/bin/reoback.pl"
    
    # Do not modify this line.
    $reoback $config

    Finally you can now test your backup

    /etc/reoback/run_reoback.sh

    or place this command in crontab

    * 19 * * * /etc/reoback/run_reoback.sh > backup.txt ;
    mail -s "automatisches Backup" This email address is being protected from spambots. You need JavaScript enabled to view it. < backup.txt

    Tips

    Depending where you visitor come from (America or Asia or Europe) it may be recommended to not start your backup during peak of visits, You can also nice the process to a lower priority

    * 19 * * * nice –19 /etc/reoback/run_reoback.sh > backup.txt ; 
    mail -s "automatisches Backup" This email address is being protected from spambots. You need JavaScript enabled to view it. < backup.txt
  • chkrootkit is a tool to locally check for signs of a rootkit. chkrootkit is a common unix-based program intended to help system administrators check their system for known rootkits. It works by using several mechanisms, including comparison of file signatures to known rootkits, checking for suspicious activity (processes listed in the proc filesystem but not in the output of the 'ps' command.
    Log to the server with ssh as root user

    Download 
    chkrootkit.
    # wget ftp://ftp.pangeia.com.br/pub/seg/pac/chkrootkit.tar.gz

    Unpack the chkrootkit you just downloaded.
    # tar xvzf chkrootkit.tar.gz

    go to that  directory
    # cd chkrootkit

    Compile
    # make sense

    Run
    # chkrootkit

     
    •Receive e-mail everyday with the result chkrootkit
    For Root user
    # crontab -e
    For any user
    # crontab -e -u username

    and add

    •0 3 * * * (./usr/sbin/chkrootkit 2>&1 | mail -s "chkrootkit output" -c This email address is being protected from spambots. You need JavaScript enabled to view it.,This email address is being protected from spambots. You need JavaScript enabled to view it. This email address is being protected from spambots. You need JavaScript enabled to view it.)

    * the correct path can be found with which chkrootkit
    This will run chkrootkit at 3:00 am every day, and e-mail the output to This email address is being protected from spambots. You need JavaScript enabled to view it. and copies to This email address is being protected from spambots. You need JavaScript enabled to view it. and This email address is being protected from spambots. You need JavaScript enabled to view it.

    False alarms:
     "Checking `bindshell'... INFECTED (PORTS: 465)" This is normal and  NOT really a rootkit.

    Nota
    If you ever get a positive alarm, you can try to remove the rootkit, but all professionals would advice you to reinstall the server from scratch, and restore a previous backup (that mean saving nothing from server as soon as the rootkit is revealed....)

    Links
    chkrootkit
  • Here is the easiest way to run a daily backup of your database using linux crontab. Thanks to our everyday increasing mailbox size (Thanks Gmail) and their nature to be quite safe for saving document, let’s use them to store the full backup of all our Mysql database!

    Requirements

    • Having a shell access to your linux box
    • Creating a user with limited mysql rights: SELECT and LOCK_TABLES is enough, For example a user backup_user with a password ChhdeqyqUzd75687fOnmYar
    • Installing Mpack: Tools for encoding/decoding MIME messages. Mpack and munpack are utilities for encoding and decoding (respectively) binary files in MIME (Multipurpose Internet Mail Extensions) format mail messages. For compatibility with older forms of transferring binary files, the munpack program can also decode messages in split-uuencoded format.

    Edit your crontab

    crontab -e

    and put inside on one line the following

    0 1 * * * /usr/bin/mysqldump -ubackup_user –pChhdeqyqUzd75687fOnmYar yourdb |
    gzip > /database_`date +'%m-%d-%Y'`.sql.gz ;
    mpack -s "Databases backup"
    -c application/gzip /database_`date +'%m-%d-%Y'`.sql.gz This email address is being protected from spambots. You need JavaScript enabled to view it.

    You can replace the word yourdb with your tablename or --all-databases to dump all database. With the above line a backup will be run at 1AM every day and sent in your mailbox.

  • Rootkit scanner is scanning tool to ensure you for about 99.9%* you're clean of nasty tools. This tool scans for rootkits, backdoors and local exploits by running tests like:
    • MD5 hash compare
    • Look for default files used by rootkits
    • Wrong file permissions for binaries
    • Look for suspected strings in LKM and KLD modules
    • Look for hidden files
    • Optional scan within plaintext and binary files
    Rootkit Hunter is released as GPL licensed project and free for everyone to use. # wget http://downloads.rootkit.nl/rkhunter-1.1.4.tar.gz
    # tar -xzvf rkhunter-1.1.4.tar.gz
    # cd rkhunter
    # ./installer.sh


    Receive e-mail everyday with the result Rootkit Hunter
    For Root user
    # crontab -e
    For any user
    # crontab -e -u username

    and add

    •0 3 * * * (./usr/local/bin/rkhunter –checkall 2>&1 | mail -s "chkrootkit output" -c This email address is being protected from spambots. You need JavaScript enabled to view it.,This email address is being protected from spambots. You need JavaScript enabled to view it. This email address is being protected from spambots. You need JavaScript enabled to view it.)

    * the correct path can be found with which rkhunter 
    This will run Rootkit Hunter at 3:00 am every day, and e-mail the output to This email address is being protected from spambots. You need JavaScript enabled to view it. and copies to This email address is being protected from spambots. You need JavaScript enabled to view it. and This email address is being protected from spambots. You need JavaScript enabled to view it.

    Nota
    If you ever get a positive alarm, you can try to remove the rootkit, but all professionals would advice you to reinstall the server from scratch, and restore a previous backup (that mean saving nothing from server as soon as the rootkit is revealed....)

    Links

    http://www.rootkit.nl/projects/rootkit_hunter.html
  • SIM is a system and services monitor for ‘SysVinit’ systems. It is designed to be intuitive and modular in nature, and to provide a clean and informative status system.
    It does this by consistently verifying that services are online, load averages are in check, and log files are at reasonable sizes. Many other SIM modules sport different and in-depth features to bring a well rounded tool to your disposal to stop otherwise common issues daunting internet hosts.

    Features:
    - Service monitoring of HTTP, FTP, DNS, SSH, MYSQL & more
    - Event tracking and alert system
    - Auto restart ability for downed services
    - Checks against network sockets & process list to ensure services are online
    - Advanced HTTP service monitoring, to prevent commonly encountered issues
    - System load monitor with customizable warnings & actions
    - Ability to auto restart system with definable critical load level
    - Priority change configurable for services, at warning or critical load level
    - Informative command line status display
    - Easily customizable configuration file
    - Auto configuration script
    - Auto cronjob setup feature
    - Simple & Informative installation script
    - Integrated auto-update feature
    - And more...

    From http://www.r-fx.org/sim.php

    Installation is one more time straightforward:

    # wget http://www.r-fx.ca/downloads/sim-current.tar.gz
    # tar xvf sim-current.tar.gz
    # cd sim-*

    The installation of sim is easily acomplished, a simple shell script named  'setup' is included with SIM. Running this script will tend to all the install tasks for SIM.

    # ./setup -i
    -i     Install
    -q     Quick install
    -u     Uninstall
    -c     Install/Uninstall cronjob

    Press "SPACE" to go to the next page when you read the licence.

    Press "RETURN" to quit

    The readme is then displayed, press "SPACE", then "RETURN"

    Ideally once SIM is configured it is best to run from a cronjob. The 'setup'
    SIM 2.5-4 <This email address is being protected from spambots. You need JavaScript enabled to view it.>
    Creating installation paths:            [##########]
    Installing SIM 2.5-4 to /usr/local/sim:         [##########]

    SIM 2.5-4 installation completed, related notes:
    Executable:             /usr/local/sim/sim
    Executable symlink:     /usr/local/sbin/sim
    Config file:            /usr/local/sim/conf.sim
    Autoconf script:        /usr/local/sim/autoconf
    Autoconf symlink:       /usr/local/sbin/sim-autoconf
    Cronjob setup:          /usr/local/sim/sim -j

    SIM 2.5-4 must now be configured for use on this system, Press
    return to run the autoconf script (/usr/local/sim/autoconf).

    SIM 2.5-4 Auto-Config Script

    All questions default to value in brackets if no answer is given. If you
    make a typo during the autoconf process, hit CTRL+C (^C) to abort and
    rerun the autoconf script (/usr/local/sim/autoconf).

    The below are general configuration options for SIM:
    press return to continue...

    Where is SIM installed ?
    [/usr/local/sim]:
    "RETURN"

    Where should the sim.log file be created ?
    [/usr/local/sim/sim.log]:
    "RETURN"

    Max size of sim.log before rotated ? (value in KB)
    [128]:
    "RETURN"

    What is the location of your kernel log ?
    Found kernel log at /var/log/messages
    "RETURN"

    Where should alerts be emailed to ? (e.g: root, user@domain)
    [root]:  This email address is being protected from spambots. You need JavaScript enabled to view it.  
    "RETURN" enter a external email, not one from the mail server domain!

    Disable alert emails after how many events, to avoid email flood ?
    (Note: events stats are cleared daily)
    [8]:
    "RETURN"

    The below are configuration options for Service modules:
    press return to continue...

    Auto-restart services found to be offline ? (true=enable, false=disable)
    [true]:
    "RETURN"

    Enforce laxed service checking ? (true=enable, false=disable)
    [true]:
    "RETURN"

    Disable auto-restart after how many downed service events ?
    (Note: events stats are cleared daily)
    [10]:
    "RETURN"

    Enable FTP service monitoring ? (true=enable, false=disable)
    [false]:
    "RETURN"

    Name of the FTP service as appears in 'ps' ?
    Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
    Found service name as proftpd

    TCP/IP port that FTP operates on ?
    [21]:
    "RETURN"

    Path to FTP service init script ?
    [/etc/init.d/proftpd]:
    "RETURN"

    Enable HTTP service monitoring ? (true=enable, false=disable)
    [false]:true
    "RETURN"

    Name of the HTTP service as appears in 'ps' ?
    Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
    Found service name as httpd

    TCP/IP port that HTTP operates on ?
    [80]:
    "RETURN"

    Path to HTTP service init script ?
    [/etc/init.d/httpd]:
    "RETURN"

    Enable DNS service monitoring ? (true=enable, false=disable)
    [false]:true
    "RETURN"

    Name of the DNS service as appears in 'ps' ?
    Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
    Found service name as named


    TCP/IP port that DNS operates on ?
    Found service port as 53

    Path to DNS service init script ?
    Found service init script at /etc/init.d/named

    Enable SSH service monitoring ? (true=enable, false=disable)
    [false]:true
    "RETURN"

    Name of the SSH service as appears in 'ps' ?
    Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
    Found service name as sshd

    TCP/IP port that SSH operates on ?
    Found service port as 22
    "RETURN"

    Path to SSH service init script ?
    Found service init script at /etc/init.d/sshd

    Enable MYSQL service monitoring ? (true=enable, false=disable)
    [false]:true
    "RETURN"

    Name of the MYSQL service as appears in 'ps' ?
    Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
    Found service name as mysqld

    TCP/IP port that MYSQL operates on ?
    Found service port as 3306

    Path to MYSQL service init script ?
    Found service init script at /etc/init.d/mysql

    Enable SMTP service monitoring ? (true=enable, false=disable)
    [false]:   
    "RETURN"

    Enable XINET service monitoring ? (true=enable, false=disable)
    [false]:true

    Name of the XINET service as appears in 'ps' ?
    Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
    Found service name as xinetd

    TCP/IP port that any XINET service operates on (e.g: pop3, 110) ?
    [110]:
    "RETURN"

    In computer networking, xinetd, the eXtended InterNET Daemon, is an open-source daemon which runs on many Unix systems and manages Internet-based connectivity. It offers a more secure extension to or version of inetd, the Internet daemon.

    xinetd features access control mechanisms such as TCP Wrapper ACLs, extensive logging capabilities, and the ability to make services available based on time. It can place limits on the number of servers that the system can start, and has deployable defence mechanisms to protect against port scanners, among other things. from WikiPedia

    Path to XINET service init script ?   seewww.xinetd.org/faq.html  and 
    Found service init script at /etc/init.d/xinetd

    Enable ENSIM service monitoring ? (true=enable, false=disable)
    [false]:
    "RETURN"

    Enable PGSQL service monitoring ? (true=enable, false=disable)
    [false]:
    "RETURN"

    The below are configuration options for Service Specific features:
    press return to continue...
    After an unclean HTTP shutdown, semaphore array's may remain allocated
    and cause the service to fall into a looping restart cycle. Using this
    feature clears semaphore arrays on HTTP restart.
    Enable semaphore cleanup ?
    [false]:
    "RETURN"

    This is an implamented feature in the http module, its purpose is to
    determine if/when the apache server locks up or otherwise stops
    responding.
    Enable URL aware monitoring ?
    [false]:
    "RETURN"

    HTTP log files can grow large and cause the service to crash
    (segfault), this feature will keep the main HTTP logs incheck.
    Enable HTTP log monitor ?
    [false]:true
    "RETURN"

    What is the location of your HTTP servers, log files ?
    (should point to a directory, not file)
    [/var/log/httpd]:/var/log/apache2

    Max size of HTTP log files, before cleared ? (value in MB)
    [300]:
    "RETURN"

    MySQL uses a /tmp symlink of its mysql.sock socket file. This
    feature verifies that the symlink exists from the main mysql.sock
    file, and if not it is recreated.
    Enable MySQL Socket correction ?
    [false]:
    "RETURN"

    The below are configuration options for System modules:
    press return to continue...

    Enable NETWORK monitoring ? (true=enable, false=disable)
    [false]:true
    "RETURN"

    interface to monitor ?
    [eth0]:
    "RETURN"

    Path to NETWORK init script ?
    Found service init script at /etc/init.d/network

    Enable LOAD monitor ? (true=enable, false=disable)
    [false]:
    "RETURN"

    Configuration completed, saving conf.sim...
    Done, conf.sim saved to /usr/local/sim.

    Now the SIM (System Integrity Monitor) has been configured! add it as cron

    # ./setup -c
    SIM 2.5-4 <This email address is being protected from spambots. You need JavaScript enabled to view it.>
    Removed SIM cronjob.
    # ./setup -c
    SIM 2.5-4 <This email address is being protected from spambots. You need JavaScript enabled to view it.>
    Installed SIM cronjob.


    if everything goes well, you can check the installation by typing:

    # /etc/init.d/mysqld stop

    This will stop mysql daemon!, You will receive an email  at the same time, showing that mysql has been stopped and restarted

    System integrity monitor on xxxxx has taken action in responce to an event. Recent event logs are enclosed below for your inspection. There has been 1 events today, if an average of 8 events is reached, e-mail alerts will be terminated for the duration of the day.

    - Events Summary:
    Total event count:   1
    Average event count: 0

    - Service Summary:
    FTP       
    [online - 0 events]
    HTTP      [online - 0 events]
    DNS       [online - 0 events]
    SSH       [online - 0 events]
    MYSQL     [
    restart success1 events]
    XINET     [online - 0 events]

    - System Summary:
    NETWORK   [eth0 - online - 0 events]

    - SIM Log:
    [07/21/07 12:10:01]: touched log file.
    [07/21/07 12:10:01]: sim.dat not found, created.
    [07/21/07 12:10:01]: no .chk modules enabled.
    [07/21/07 12:15:03]: no .chk modules enabled.
    [07/21/07 12:20:01]: no .chk modules enabled.
    [07/21/07 12:25:01]: NETWORK is online.
    [07/21/07 12:25:01]: FTP service is offline.
    [07/21/07 12:25:01]: FTP service is offline.
    [07/21/07 12:25:01]: FTP restart failed, could not find /etc/init.d/proftpd.
    [07/21/07 12:25:01]: FTP restart failed, could not find /etc/init.d/proftpd.
    [07/21/07 12:25:01]: HTTP service is online.
    [07/21/07 12:25:01]: DNS service is online.
    [07/21/07 12:25:01]: SSH service is online.
    [07/21/07 12:25:01]: MYSQL service is online.
    [07/21/07 12:25:01]: XINET service is online.