Suji's blog: May 2007

Tuesday, May 01, 2007

Format of passwd and shadow files

Format of the /etc/passwd file

A non-shadowed /etc/passwd file has the following format:

username:passwd:UID:GID:full_name:directory:shell

Where:

username: The user (login) name
passwd: The encoded password
UID: Numerical user ID
GID: Numerical default group ID
full_name: The user's full name - Actually this field is called the GECOS (General Electric Comprehensive Operating System) field and can store information other than just the full name. The Shadow commands and manual pages refer to this field as the comment field.
directory: User's home directory (Full pathname)
shell: User's login shell (Full Pathname)

For example:

username:Npge08pfz4wuk:503:100:Full Name:/home/username:/bin/sh

Where Np is the salt and ge08pfz4wuk is the encoded password. The encoded salt/password could just as easily have been kbeMVnZM0oL7I and the two are exactly the same password. There are 4096 possible encodings for the same password. (The example password in this case is 'password', a really bad password).

Once the shadow suite is installed, the /etc/passwd file would instead contain:

username:x:503:100:Full Name:/home/username:/bin/sh

The x in the second field in this case is now just a place holder. The format of the /etc/passwd file really didn't change, it just no longer contains the encoded password. This means that any program that reads the /etc/passwd file but does not actually need to verify passwords will still operate correctly.

The passwords are now relocated to the shadow file (usually /etc/shadow file).

Format of the shadow file

The /etc/shadow file contains the following information:

username:passwd:last:may:must:warn:expire:disable:reserved

Where:

username: The User Name
passwd: The Encoded password
last: Days since Jan 1, 1970 that password was last changed
may: Days before password may be changed
must: Days after which password must be changed
warn: Days before password is to expire that user is warned
expire: Days after password expires that account is disabled
disable: Days since Jan 1, 1970 that account is disabled
reserved: A reserved field

The previous example might then be:

username:Npge08pfz4wuk:9479:0:10000::::

Configuring Quota on Linux

Configuration of disk usage quotas on Linux - Perform the following as root:

Edit file /etc/fstab to add qualifier "usrquota" or "grpquota" to the partition. The following file system mounting options can be specified in /etc/fstab: grpquota, noquota, quota and usrquota. (These options are also accepted by the mount command but ignored.) The filesystem when mounted will show up in the file /etc/mtab, the list of all currently mounted filesystems.)
- To enable user quota support on a file system, add "usrquota" to the fourth field containing the word "defaults".
  ...
  /dev/hda2 /home ext3 defaults,usrquota 1 1
  ...
- Replace "usrquota" with "grpquota", should you need group quota support on a file system.
  ...
  /dev/hda2 /home ext3 defaults,grpquota 1 1
  ...
- Need both user quota and group quota support on a file system?
  ...
  /dev/hda2 /home ext3 defaults,usrquota,grpquota 1 1
  ...
  This enables user and group quotas support on the /home file system.

touch /partition/aquota.user
where the partition might be /home or some partition defined in /etc/fstab.
then
chmod 600 /partition/aquota.user
The file should be owned by root. Quotas may also be set for groups by using the file aquota.group
Quota file names:
- Quota Version 2 (Linux 2.4/2.6 kernel: Red Hat 7.1+/8/9,FC 1-3): aquota.user, aquota.group
- Quota Version 1 (Linux 2.2 kernel: Red Hat 6, 7.0): quota.user, quota.group
The files can be converted/upgraded using the convertquota command.

Re-boot or re-mount file partition with quotas.
- Re-boot: shutdown -r now
- Re-mount partition: mount -o remount /partition
After re-booting or re-mounting the file system, the partition will show up in the list of mounted filesystems as having quotas. Check /etc/mtab:
...
/dev/hda5 / ext3 rw,usrquota 0 0
...
quotacheck -vgum /partition
or
quotacheck -vguma
- For example (Linux kernel 2.4+: Red Hat 7.1+, Fedora): quotacheck -vguma
```
quotacheck: WARNING -  Quotafile //aquota.user was probably truncated. ...
quotacheck: Scanning /dev/hda5 [/] done
quotacheck: Checked 9998 directories and 179487 files
  
```
- For example (Linux kernel 2.2: Red Hat 6/7.0): quotacheck -v /dev/hda6
  System response:
```
      Scanning /dev/hda6 [/home] done
    Checked 444 directories and 3136 files
    Using quotafile /home/quota.user     
    
```
Quotacheck is used to scan a file system for disk usages, and updates the quota record file "quota.user/aquota.user" to the most recent state. It is recommended thet quotacheck be run at bootup (part of Redhat default installation)
Man page: quotacheck - scan a filesystem for disk usage, create, check and repair quota files
quotaon -av
System Response: /dev/hda6: user quotas turned on
quotaon - enable disk quotas on a file system.
quotaoff - turn off disk quotas for a file system.
Man page: quotaon - turn filesystem quotas on and off

edquota -u user_id
Edit directly using vi editor commands. (See below for more info.)
For example: edquota -u user1

System Response (RH 7+):
```
Disk quotas for user user1 (uid 501):
Filesystem                   blocks       soft       hard     inodes     soft     hard
/dev/hda5                      1944          0          0        120        0        0
    
```
- blocks: 1k blocks
- inodes: Number of entries in directory file
- soft: Max number of blocks/inodes user may have on partition before warning is issued and grace persiod countdown begins.
  If set to "0" (zero) then no limit is enforced.
- hard: Max number of blocks/inodes user may have on partition.
  If set to "0" (zero) then no limit is enforced.

System Response (RH 6):

           Quotas for user user1:
         /dev/sdb6: blocks in use: 56, limits (soft = 0, hard = 0)
                 inodes in use: 50, limits (soft = 0, hard = 0)

Something failed if you get the response:

           /dev/sdb6: blocks in use: 0, limits (soft = 0, hard = 0)
                 inodes in use: 0, limits (soft = 0, hard = 0)

Edit limits:

           Quotas for user user1:
         /dev/hda6: blocks in use: 992, limits (soft = 50000, hard = 55000)
                 inodes in use: 71, limits (soft = 10000, hard = 11000)

If editing group quotas: edquota -g group_name

Man page: edquota - edit user quotas

List quotas:
quota -u user_id
For example: quota -u user1
System response:
```
Disk quotas for user user1 (uid 501):
   Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
    /dev/hda6     992   50000   55000              71   10000   11000
    
```
If this does not respond similar to the above, then restart the computer: shutdown -r now
Man page: quota - display disk usage and limits

Quota Reports

Report on all users over quota limits: quota -q

Quota summary report: repquota -a

*** Report for user quotas on device /dev/hda5
Block grace time: 7days; Inode grace time: 7days
                       Block limits                File limits
User            used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      -- 4335200       0       0         181502     0     0
bin       --   15644       0       0            101     0     0
...
user1     --    1944       0       0            120     0     0

No limits shown with this user as limits are set to 0.

Man page: repquota - summarize quotas for a filesystem.

Cron:
Quotacheck should scan the file system via cronjob periodically (say, every week?). Add a script to the /etc/cron.weekly/ directory.
File: /etc/cron.weekly/runQuotacheck

Linux Kernel 2.4: Red Hat 7.1 - Fedora Core 3:
#!/bin/bash
/sbin/quotacheck -vguma
Linux Kernel 2.2: Red Hat 6/7.0:
#!/bin/bash
/sbin/quotacheck -v -a

(Remember to chmod +x /etc/cron.weekly/runQuotacheck)

EdQuota Notes:

The "edquota" command puts you into a "vi" editing mode so knowledge of the "vi" editor is necessary. Another editor may be specified with the EDITOR environment variable. You are NOT editing the quota.user file directly. The /partition/quota.user or quota.group file is a binary file which you do not edit directly. The command edquota gives you an ascii interface with the text prepared for you. When you ":wq" to save the file from the vi session, it is converted to binary by the edquota command and stored in the quota.user file.

Assigning quota for a bunch of users with the same value. To rapidly set quotas for all users, on my system to the same value as user user1, I would first edit user user1's quota information by hand, then execute:

  edquota -p user1 `awk -F: '$3 &gt; 499 {print $1}' /etc/passwd`

This assumes that the user uid's start from 500 and increment upwards. "blocks in use" is the total number of blocks (in kilobytes) a user has comsumed on a partition. "inodes in use" is the total number of files a user has on a partition.

edquota options:

Option	Description
-r -m	Edit quotas on remote server using RPC. Remote server must be configured with the daemon `rpc.rquotad`
-u	Edit user quota
-g	Edit group quota
-p user-id	Duplicate the quotas based on existing prototype user
-F format -F vfsold -F vfsv0 -F rpc -F xfs	Format: vfsold - version 1 vfsv0 - version 2 rpc - quotas over NFS xfs - quotas for XFS filesystem
-f /file-system	Perform on specified filesystem. Default is to apply on all filesystems with quotas
-t	Edit the soft time limits for each filesystem.
-T	Edit time for user/group when softlimit is enforced. Specify number and unit or "unset"

Soft Limit and Hard Limits:

Soft limit indicates the maximum amount of disk usage a quota user has on a partition. When combined with "grace period", it acts as the border line, which a quota user is issued warnings about his impending quota violation when passed. Hard limit works only when "grace period" is set. It specifies the absolute limit on the disk usage, which a quota user can't go beyond his "hard limit".

Grace Period:

"Grace Period" is configured with the command "edquota -t", "grace period" is a time limit before the "soft limit" is enforced for a file system with quota enabled. Time units of sec(onds), min(utes), hour(s), day(s), week(s), and month(s) can be used. This is what you'll see with the command "edquota -t":

System response:

Linux Kernel 2.4+: Red Hat 7.1+/Fedora:

Grace period before enforcing soft limits for users:
Time units may be: days, hours, minutes, or seconds
 Filesystem             Block grace period     Inode grace period
 /dev/hda5                     7days                  7days

Linux Kernel 2.2: Red Hat 6/7.0:

Time units may be: days, hours, minutes, or seconds
Grace period before enforcing soft limits for users:
/dev/hda2: block grace period: 0 days, file grace period: 0 days

Change the 0 days part to any length of time you feel reasonable. A good choice might be 7 days (or 1 week).

Quota files: (non-XFS file systems)

The edquota command will create/edit the quota file at the root of the file system. (See /etc/mtab for the list of the currently mounted filesystems.)

Version 2: aquota.user, aquota.group
Version 1: quota.user, quota.group

Self Signed SSL certificates

Use self-signed certificates to test single systems, such as a test web server. Self-signed certificates become impractical in any other case. A local CA, while more complex to setup, reduces the number of keys that need to be distributed for verification, and properly replicates a real world certificate environment.

Creation of certificates requires the openssl utility. This command should be part of an OpenSSL installation, though may be installed out of the standard search path in /usr/local/ssl/bin or elsewhere.

$ which openssl
/usr/bin/openssl

Generate the Rivest, Shamir and Adleman (RSA) key

OpenSSL can generate a Digital Signature Algorithm (DSA) key (with the gendsa option), though for compatibility RSA keys are most frequently used. Learn more about the genrsa option to openssl.

$ openssl genrsa 1024 > host.key
$ chmod 400 host.key

Modern systems should provide a random device and sufficient entropy for key generation. The data in the host.key file must be protected, as anyone with this information can decrypt traffic encrypted with this key.

Create the Certificate

Learn more about the req option to openssl. The -new, -x509 and -nodes arguments are required to create an unencrypted certificate. The -days argument specifies how long the certificate will be valid for.

$ openssl req -new -x509 -nodes -sha1 -days 365 -key host.key > host.cert

Questions may be asked to fill out the certificate’s x509 attributes. The answers should be adjusted for the locale:

Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Washington
Locality Name (eg, city) []:Seattle
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Sial.org
Organizational Unit Name (eg, section) []:
Common Name (eg, YOUR name) []:mail.example.org
Email Address []:postmaster@example.org

The Common Name field usually must exactly match the hostname of the system the certificate will be used on; otherwise, clients should complain about a certificate to hostname mismatch.

The certificate data in the host.cert file does not need to be protected like the private key file does. In fact, it will likely need to be transferred to all the client systems that need to verify the key of the server being connected to. If this is the case, setup a CA, and distribute the signing certificate to the clients instead of each self-signed certificate.

Extract Metadata (Optional)

Optionally, various certificate metadata can be saved for quick reference, for example to verify the key fingerprint. Learn more about the x509 option to openssl.

$ openssl x509 -noout -fingerprint -text <> host.info

Combine Key and Certificate Data (Optional)

Some applications may require that the key and certificate data be in a single file. I recommend keeping the key and certificate data separate if possible, as the key data needs to be protected, and the certificate data available to all. Combining the data means the resulting file must be protected like a key file.

$ cat host.cert host.key > host.pem \ && rm host.key
$ chmod 400 host.pem

The host.cert certificate data will need to be exported to client systems for use in testing.

The openssl.cnf file

Localize the system openssl.cnf to include relevant X509 attributes of the certificate. This will save typing and avoid errors when creating certificates. The location of this file varies by system.

$ grep Name_default /etc/ssl/openssl.cnf
countryName_default = US
stateOrProvinceName_default = Washington
0.organizationName_default = Sial.org
#1.organizationName_default = World Wide Web Pty Ltd
#organizationalUnitName_default =

Suji's blog