Build ubuntu server from scratch

SSH

• Install SSH server

apt-get install openssh-server 

sshd_config is the configuration file for the OpenSSH server. ssh_config is the configuration file for the OpenSSH client. Make sure not to get them mixed up.

• Config file location

/etc/ssh/sshd_config

• Restart the SSH service

systemctl restart ssh

mount

• Auto mount disk when ubuntu start
  Decide which partitions to mount

System name English name Linux type W95 FAT32 Microsoft FAT32 vfat W95 FAT32 (LBA) Microsoft FAT32 vfat W95 FAT16 (LBA) Microsoft FAT16 vfat W95 Ext’d (LBA) Microsoft extended partition Not used NTFS volume set Microsoft NTFS ntfs NTFS volume set Microsoft NTFS with read-write access ntfs-3g Apple_HFS Apple HFS hfsplus

• option example

To learn more about options, type ‘man mount’.

Description Accessible by everyone Accessible by a subset of users** FAT(16/32) partition user,auto,fmask=0111,dmask=0000 user,auto,fmask=0177,dmask=0077,uid=1000 NTFS partition* rw,auto,user,fmask=0111,dmask=0000 rw,user,auto,fmask=0177,dmask=0077,uid=1000 Apple Partition user,auto,file_umask=0111,dir_umask=0000 user,auto,file_umask=0177,dir_umask=0077,uid=1000

• If you want write access to your file system, you should set the filesystem type to ‘ntfs-3g’ instead of ‘ntfs’. You may need to install the package ‘ntfs-3g’ for this to work, so make sure it is installed before you use ntfs-3g.
• uid=1000 restricts access to the user created while installing Ubuntu. 1001 is the user created after that, and so forth. gid=# may be used with or in place of uid to grant access to a group. However, group and user enumeration is beyond the scope of this article.

vi /etc/fstab

UUID=519CB82E5888AD0F  /media/Data  ntfs-3g  user,auto,file_umask=0111,dir_umask=0000  0 0 

you can find the UUID by running the following command

blkid

Auto mount knowledge link

samba

Please refer the link

sudo apt-get install sambasudo useradd USERNAME --shell /bin/falsesudo vi /etc/samba/smb.conf

add the following lines at the end of the config file

Once "smb.conf" has loaded, add this to the very end of the file:[<folder_name>]path = /home/<user_name>/<folder_name>valid users = <user_name>read only = no

sudo smbpasswd -a <user_name>sudo service smbd restart

Hdparm

sudo apt-get install hdparm

Make sure your drive supports hd parm, if you have multiple hard drives it could be /dev/sdb or /dev/sdb – the command blkid will show you all disk drives connected.

sudo hdparm -y /dev/sda

You should get output like this indicating a successful standby command

/dev/sda: issuing standby command

Check if your drive supports write cache

sudo hdparm -I /dev/sda | grep 'Write cache'

If you see a * (asterix) then you are good to go.

*    Write cache

If you don’t see a star (asterix) then write cache is not possible for your drive

Time to make hdparm configurations permanent edit the configuration file

sudo nano /etc/hdparm.conf

The spindown_time value is multiplied by 5 and you have the total time in seconds. So a value of 120 yields 10 minutes (120*5=600).

Enable write cache and spindown time by adding this text to the bottom of the file

/dev/sda {write_cache = onspindown_time = 120}

Systemd

Refer this link
Unbuntu systemd

• Example systemd service

[Unit]Description=Job that runs the foo daemonDocumentation=man:foo(1)[Service]Type=forkingEnvironment=statedir=/var/cache/fooExecStartPre=/usr/bin/mkdir -p ${statedir}ExecStart=/usr/bin/foo-daemon --arg1 "hello world" --statedir ${statedir}[Install]WantedBy=multi-user.target

To determine which init daemon you are currently booting with, run:

ps -p1 | grep systemd && echo systemd || echo upstart

• systemctl usages:

systemctl statussystemctl list-unitssystemctl --failedsystemctl list-unit-filessystemctl is-enabled unitsystemctl enable unit

• journalctl usages:

journalctl -bjournalctl --since="2012-10-30 18:17:16"journalctl --since "20 min ago"journalctl /usr/lib/systemd/systemd

SSL Certificate

• To generate the keys for the Certificate Signing Request (CSR) run the following command from a terminal prompt:

openssl genrsa -des3 -out server.key 2048

• Now create the insecure key, the one without a passphrase, and shuffle the key names:

openssl rsa -in server.key -out server.key.insecuremv server.key server.key.securemv server.key.insecure server.key

• To create the CSR, run the following command at a terminal prompt:

openssl req -new -key server.key -out server.csr

• To create the self-signed certificate, run the following command at a terminal prompt:

openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

nginx

Location config

• The = modifier
  The requested document URI must match the specified pattern exactly. The pattern
  here is limited to a simple literal string; you cannot use a regular expression.
• No modifier
  The requested document URI must begin with the specified pattern. You may not
  use regular expressions.
• The ~ modifier
  The requested URI must be a case-sensitive match to the specified regular expression
• The ~* modifier
  The requested URI must be a case-insensitive match to the specified regular expression.
• The ^~ modifier
  Similar to the no-symbol behavior, the location URI must begin with the specified
  pattern. The difference is that if the pattern is matched, Nginx stops searching for
  other patterns (read the section below about search order and priority).
• The @ modifier
  Defines a named location block. These blocks cannot be accessed by the client,
  but only by internal requests generated by other directives, such as try_files or
  error_page.
• Search order
  Nginx will search for matching patterns in a
  specific order:
  
  1. location blocks with the = modifier: If the specified string exactly matches
     the requested URI, Nginx retains the location block.
  2. location blocks with no modifier: If the specified string exactly matches the
     requested URI, Nginx retains the location block.
  3. location blocks with the ^~ modifier: If the specified string matches the
     beginning of the requested URI, Nginx retains the location block.
  4. location blocks with ~ or ~* modifier: If the regular expression matches the
     requested URI, Nginx retains the location block.
  5. location blocks with no modifier: If the specified string matches the
     beginning of the requested URI, Nginx retains the location block.
     In that extent, the ^~ modifier begins to make sense, and we can envision cases
     where it becomes useful.

Script

• nginx enable/disable script

#!/bin/bash###  File:#    nginx_modsite#  Description:#    Provides a basic script to automate enabling and disabling websites found#    in the default configuration directories:#      /etc/nginx/sites-available and /etc/nginx/sites-enabled#    For easy access to this script, copy it into the directory:#      /usr/local/sbin#    Run this script without any arguments or with -h or --help to see a basic#    help dialog displaying all options.### Copyright (C) 2010 Michael Lustfield <[e-mail]># Redistribution and use in source and binary forms, with or without# modification, are permitted provided that the following conditions# are met:# 1. Redistributions of source code must retain the above copyright#    notice, this list of conditions and the following disclaimer.# 2. Redistributions in binary form must reproduce the above copyright#    notice, this list of conditions and the following disclaimer in the#    documentation and/or other materials provided with the distribution.## THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE# ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF# SUCH DAMAGE.### Default Settings##NGINX_CONF_FILE="$(awk -F= -v RS=' ' '/conf-path/ {print $2}' <<< $(nginx -V 2>&1))"NGINX_CONF_DIR="${NGINX_CONF_FILE%/*}"NGINX_SITES_AVAILABLE="$NGINX_CONF_DIR/sites-available"NGINX_SITES_ENABLED="$NGINX_CONF_DIR/sites-enabled"SELECTED_SITE="$2"### Script Functions##ngx_enable_site() {    [[ ! "$SELECTED_SITE" ]] &&        ngx_select_site "not_enabled"    [[ ! -e "$NGINX_SITES_AVAILABLE/$SELECTED_SITE" ]] &&         ngx_error "Site does not appear to exist."    [[ -e "$NGINX_SITES_ENABLED/$SELECTED_SITE" ]] &&        ngx_error "Site appears to already be enabled"    ln -sf "$NGINX_SITES_AVAILABLE/$SELECTED_SITE" -T "$NGINX_SITES_ENABLED/$SELECTED_SITE"    ngx_reload}ngx_disable_site() {    [[ ! "$SELECTED_SITE" ]] &&        ngx_select_site "is_enabled"    [[ ! -e "$NGINX_SITES_AVAILABLE/$SELECTED_SITE" ]] &&        ngx_error "Site does not appear to be \'available\'. - Not Removing"    [[ ! -e "$NGINX_SITES_ENABLED/$SELECTED_SITE" ]] &&        ngx_error "Site does not appear to be enabled."    rm -f "$NGINX_SITES_ENABLED/$SELECTED_SITE"    ngx_reload}ngx_list_site() {    echo "Available sites:"    ngx_sites "available"    echo "Enabled Sites"    ngx_sites "enabled"}### Helper Functions##ngx_select_site() {    sites_avail=($NGINX_SITES_AVAILABLE/*)    sa="${sites_avail[@]##*/}"    sites_en=($NGINX_SITES_ENABLED/*)    se="${sites_en[@]##*/}"    case "$1" in        not_enabled) sites=$(comm -13 <(printf "%s\n" $se) <(printf "%s\n" $sa));;        is_enabled) sites=$(comm -12 <(printf "%s\n" $se) <(printf "%s\n" $sa));;    esac    ngx_prompt "$sites"}ngx_prompt() {    sites=($1)    i=0    echo "SELECT A WEBSITE:"    for site in ${sites[@]}; do        echo -e "$i:\t${sites[$i]}"        ((i++))    done    read -p "Enter number for website: " i    SELECTED_SITE="${sites[$i]}"}ngx_sites() {    case "$1" in        available) dir="$NGINX_SITES_AVAILABLE";;        enabled) dir="$NGINX_SITES_ENABLED";;    esac    for file in $dir/*; do        echo -e "\t${file#*$dir/}"    done}ngx_reload() {    read -p "Would you like to reload the Nginx configuration now? (Y/n) " reload    [[ "$reload" != "n" && "$reload" != "N" ]] && invoke-rc.d nginx reload}ngx_error() {    echo -e "${0##*/}: ERROR: $1"    [[ "$2" ]] && ngx_help    exit 1}ngx_help() {    echo "Usage: ${0##*/} [options]"    echo "Options:"    echo -e "\t<-e|--enable> <site>\tEnable site"    echo -e "\t<-d|--disable> <site>\tDisable site"    echo -e "\t<-l|--list>\t\tList sites"    echo -e "\t<-h|--help>\t\tDisplay help"    echo -e "\n\tIf <site> is left out a selection of options will be presented."    echo -e "\tIt is assumed you are using the default sites-enabled and"    echo -e "\tsites-disabled located at $NGINX_CONF_DIR."}### Core Piece##case "$1" in    -e|--enable)    ngx_enable_site;;    -d|--disable)   ngx_disable_site;;    -l|--list)  ngx_list_site;;    -h|--help)  ngx_help;;    *)      ngx_error "No Options Selected" 1; ngx_help;;esac

shellinabox

• Setup
  sudo apt-get install shellinabox

then navigate to https://yourcomputername:4200

• Change the default config

sudo gedit /etc/default/shellinaboxSHELLINABOX_ARGS="--no-beep --localhost-only --disable-ssl"

• Restart the service
  sudo invoke-rc.d shellinabox restart

• Nginx config

  location /shellinabox/ {    rewrite ^/shellinabox/(.*) /$1 break;    proxy_pass http://127.0.0.1:4200;    proxy_read_timeout 90;  }

SNI proxy

# Install required packagessudo apt-get install autotools-dev cdbs debhelper dh-autoreconf dpkg-dev gettext libev-dev libpcre3-dev libudns-dev pkg-config fakeroot git# Clone sniproxy repo from Githubgit clone https://github.com/dlundquist/sniproxy.git# Compile and create the packagecd sniproxy./autogen.sh && dpkg-buildpackage# Install the packagesudo dpkg -i ../sniproxy_*_*.deb

• Configuration location

/etc/sniproxy.conf

# sniproxy.conf# Setup for sharing port 443 with Sandstormuser daemonpidfile /var/run/sniproxy.piderror_log {    syslog daemon    priority notice}listen 443 {    proto tls    table https_hosts    fallback 127.0.0.1:7443    access_log {        filename /var/log/sniproxy/https_access.log        priority notice    }}table https_hosts {    .*\.sandcats\.io 127.0.0.1:6443}

To make SNI proxy automatically startup on boot up

sudo update-rc.d sniproxy enable

Detail setting is here

Important tips
Usually sniproxy will work with nginx, in seafile or sandstorm official sites’ documents, by default, nginx is deployed with these services on the same server. This saves a lot of configuration effort. But, if your nginx is deployed to another independent server, we need to configure the nginx with disabling the port_in_redirect parameter in location context. This will help to remove the port in url, such as 7443 which is sent from sniproxy.

transmission

• Add Transmission PPA Repository

add-apt-repository ppa:transmissionbt/ppaapt-get update

• Install

apt-get install transmission-cli transmission-common transmission-daemon

• Config

service transmission-daemon stop

/var/lib/transmission-daemon/info/settings.json

• You need to modify the username/password, whitelist, default file directory and unmask parameters.

"rpc-password": "{62b16db87b89a91dd49a5110a7cafc06d20eb4f2wtK6kqPj","rpc-username": "transmission",----------"rpc-whitelist": "127.0.0.1,192.168.*.*",----------"umask": 2,

• Web interface
  http://server-ip:9091

Unbuntu transmission installation

Note
Need to setup the forward port on router, port 51413. Search how to port forward

• Detail configuration item

Bandwidth    alt-speed-enabled: Boolean (default = false, aka 'Turtle Mode')    Note: Clicking the "Turtle" in the gui when the scheduler is enabled, will only temporarily remove the scheduled limit until the next cycle.    alt-speed-up: Number (KB/s, default = 50)    alt-speed-down: Number (KB/s, default = 50)    speed-limit-down: Number (KB/s, default = 100)    speed-limit-down-enabled: Boolean (default = false)    speed-limit-up: Number (KB/s, default = 100)    speed-limit-up-enabled: Boolean (default = false)    upload-slots-per-torrent: Number (default = 14)

flexget

Linux installation
For the above steps, if you want to use plugins, please note:
virtualenv --system-site-packages ~/flexget/

To have flexget run as a system unit.

[Unit](/Unit)Description=Flexget DaemonAfter=network.target[Service](/Service)Type=simpleUser=daemonGroup=daemonUMask=000WorkingDirectory=/etc/flexgetExecStart=/usr/bin/flexget daemon startExecStop=/usr/bin/flexget daemon stopExecReload=/usr/bin/flexget daemon reload[Install](/Install)WantedBy=multi-user.target

sudo mkdir /etc/flexgetsudo chown daemon:daemon /etc/flexget

You can now place your config.yml file in the /etc/flexget directory.

Enable or disable Flexget at boot using :

sudo systemctl enable flexgetsudo systemctl disable flexget

Read the systemd log:

journalctl --u flexget

config.yml

tasks:  pt-task:    rss: http://mysite.com/myfeed.rss    accept_all: yes    exists: /some/download/folder    transmission:      host: localhost      port: 9091      username: myusername      password: mypassword  cleanseed:       from_transmission:      host: localhost      port: 9091      username: myusername      password: mypassword         onlycomplete: yes    clean_transmission:      host: localhost      port: 9091      username: myusername      password: mypassword      transmission_seed_limits: yes (From flexget 1.2.190 and up you need to set transmission_seed_limits: yes to get the same behaviour as previous version of this plugin or torrents may not be removed when completed.)      finished_for: 1 days    disable: detailsschedules:  - tasks: pt-task (here we can't use single quotaion mark if we use full task name)    interval:      minutes: 15  - task: cleanseed    interval:      hours: 4

• Transmission plugin config
• Plugin list
• Clean transmission plugin config

clean_transmission:  host: localhost  port: 9091  username: myusername  password: mypassword  finished_for: 2 hours  tracker: nyaa|animebytes  preserve_tracker: alpharatio|32pagdisable: details

seafile

• Please refer this link
• after seahub (web page) starts up, you have to modify the SERVICE_URL and FILE_SERVER_ROOT. For the site is behind nginx, FILE_SERVER_ROOT need to be configured as www.mydomain.com/seafhttp

• Don’t forget to modify the file path in location section namedwith /media of nginx config file, if not, it could lead to the css file can’t be loaded correctly.

• Create systemd service files, change ${seafile_dir} to your seafile installation location and seafile to user, who runs seafile (if appropriate). Then you need to reload systemd’s daemons: systemctl daemon-reload.
  Create systemd service file /etc/systemd/system/seafile.service

[Unit]Description=Seafile# add mysql.service or postgresql.service depending on your database to the line belowAfter=network.target[Service]Type=oneshotExecStart=${seafile_dir}/seafile-server-latest/seafile.sh startExecStop=${seafile_dir}/seafile-server-latest/seafile.sh stopRemainAfterExit=yesUser=seafileGroup=seafile[Install]WantedBy=multi-user.target

• Create systemd service file /etc/systemd/system/seahub.service

[Unit]Description=Seafile hubAfter=network.target seafile.service[Service]# change start to start-fastcgi if you want to run fastcgiExecStart=${seafile_dir}/seafile-server-latest/seahub.sh startExecStop=${seafile_dir}/seafile-server-latest/seahub.sh stopUser=seafileGroup=seafileType=oneshotRemainAfterExit=yes[Install]WantedBy=multi-user.target

• If seafile and nginx runs on different server, we can simplify the nginx configuration as following:

location / {         proxy_pass http://serverip:8000;         client_max_body_size 0;         proxy_connect_timeout  36000s;         proxy_read_timeout  36000s;         access_log      /var/log/nginx/seahub.access.log;         error_log       /var/log/nginx/seahub.error.log;     }     location /seafhttp {        rewrite ^/seafhttp(.*)$ $1 break;        proxy_pass http://serverip:8082;        client_max_body_size 0;        proxy_connect_timeout  36000s;        proxy_read_timeout  36000s;        proxy_send_timeout  36000s;        send_timeout  36000s;     }

at the same time, we need to modify the config in seafile web, change SERVICE_URL to local ip and port, change FILE_SERVER_ROOT to domain name/seafhttp

Use systemctl enable seafile.service and seahub.service.

Backup and Restore Ubuntu

Backup reference link
- Backup Command

tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys --exclude=/media /

or

cd / 

The following is an exemplary command of how to archive your system.

tar -cvpzf backup.tar.gz --exclude=/backup.tar.gz --one-file-system / 

–one-file-system - Do not include files on a different filesystem. If you want other filesystems, such as a /home partition, or external media mounted in /media backed up, you either need to back them up separately, or omit this flag. If you do omit this flag, you will need to add several more –exclude= arguments to avoid filesystems you do not want. These would be /proc, /sys, /mnt, /media, /run and /dev directories in root. /proc and /sys are virtual filesystems that provide windows into variables of the running kernel, so you do not want to try and backup or restore them. /dev is a tmpfs whose contents are created and deleted dynamically by udev, so you also do not want to backup or restore it. Likewise, /run is a tmpfs that holds variables about the running system that do not need backed up.

• Restore Command

tar xvpfz backup.tar.gz -C /

• If you change the hard drive, the uuid is changed, the restore will stop the system booting. There are two places need to use uuid, one is /etc/fstab, the other is /boot/grub/grub.cfg. If you use boot cd, you have to add /media/xxxx/ to the path to edit these two files. Use gksudo gedit to modify the files.

• Or before restore the backup, you can backup the above two files by changing the filename to .bak. After restore, change back the two files’ name to original.

  After reboot into system, you’d better run the following command.

sudo update-grub

• Restore Grub
  
  1. Pop in the Live CD, boot from it until you reach the desktop.
  2. Open a terminal window or switch to a tty.
  3. Type “grub”
  4. Type “root (hd0,6)”, or whatever your harddisk + boot partition numbers are (my /boot is at /dev/sda7, which translates to hd0,6 for grub).
  5. Type “setup (hd0)”, ot whatever your harddisk nr is.
  6. Quit grub by typing “quit”.
  7. Reboot.

Desktop crash restore

1. Try to open a terminal with Ctrl+Alt+T.This may not work but you can try right clicking on the desktop and selecting "Open terminal here." Otherwise, you may need to change to a "hard" terminal by pressing Ctrl+Alt+F1 and log in.2. Install compizconfig-settings-manager by running

    sudo apt-get install compizconfig-settings-manager

3. Then run it with this:

    DISPLAY=:0 ccsm &

The first part tells the terminal which display you want it to load on (otherwise it won't have a clue)4. If you switched to a TTY in step 1, switch back to the graphical server by pressing Ctrl+Alt+F7 (or Ctrl+Alt+F8 sometimes).There there should be a CompizConfig Settings Manager waiting for you.5. Find the Unity plugin. Enable it. You will be asked "Ubuntu Unity Plugin requires the plugin OpenGL. Enable Ubuntu Unity Plugin / Enable OpenGL"6. Everything should spring into life but if it doesn't, you might have to restart. You can do that by going back to the terminal and running 

sudo reboot.

If you get to step 5 and don’t see unity on the list, try this: sudo apt-get -f install && sudo apt-get –reinstall install unity

Another way to reinstall the gnome desktop.

Try:

sudo apt-get remove ubuntu-desktopsudo apt-get remove ubuntu-gnome-desktop

Do not restart. This could effectively leave your system without GUI.

sudo apt-get install ubuntu-gnome-desktopsudo apt-get autoremove

This will install all the missing GNOME dependencies.

Maintain and audit

Apply the system updates:

sudo apt-get dist-upgrade

Check the login history:

sudo less /var/log/auth.log