Tor Exit Full Setup

Basic Setup

  • we currently use Debian Jessie; Ubuntu LTS is another good valid option
  • git: we keep some scripts, configuration files and templates in a git repository. We clone the git repository to the local system and use symlinks when possible. For easier re-use in parts, the documentation below uses direct links to the configuration files at github.

SSH key authentication only

  • ssh config with password auth and PAM disabled, no root login, x11forwarding disabled
  • change port to some port of your chosing above 1024 (when using multiple IPs: bind to one IP only)
ssh-copy-id -p $SSH_PORT user@server

iptables firewall

This config by defaults allows world access to the SSH port and Tor ports 80,443. Think about limiting that to an IP (range) you can connect from, or use portknocking. Also, this config is optimized for high bandwidth relays: in order to avoid the conntrack module, it allows all UDP in!

cd /etc
wget https://raw.githubusercontent.com/torservers/server-config-templates/master/iptables.test.rules
chmod 600 iptables.test.rules
vi iptables.test.rules # update at least SSH port
iptables-restore < iptables.test.rules
  • test if you can still connect in another session :)
iptables-save > iptables.rules
chmod 600 iptables.rules
cd /etc/network/if-pre-up.d/
wget https://raw.githubusercontent.com/torservers/server-config-templates/master/iptables-restore
chmod +x iptables-restore

some useful defaults

# configure hostname
hostname yourservername.xyz
vi /etc/hostname # also use yourservername.xyz
vi /etc/hosts # update to yourservername.xyz yourservername

# disable debian default that pulls in recommended packages:
cd /etc/apt/apt.conf.d
wget https://raw.githubusercontent.com/torservers/server-config-templates/master/06norecommends

apt-get update && apt-get full-upgrade
apt-get install sudo git less htop nload screen \
ntp apticron vnstat logcheck logcheck-database lsb-release
apt-get remove --purge portmap

sed -i -e 's/^# DIFF_ONLY/DIFF_ONLY/' /etc/apticron/apticron.conf # make apticron send diffs only
vnstat -u -i eth0 # setup vnstat for correct interface
sudoedit /etc/resolv.conf # remove search line
cd /etc
mv aliases aliases.dist
wget https://raw.githubusercontent.com/torservers/server-config-templates/master/aliases
sed -i 's/your@email.address/actual@email.address/' /etc/aliases
newaliases

unattended upgrades

We upgrade from all available package sources, let it reboot if necessary, and send mail on errors. A reasonable configuration could be to limit upgrades to the security sources. (see comments in 50unattended-upgrades)

apt-get install unattended-upgrades
wget -p -O /etc/apt/apt.conf.d/50unattended-upgrades https://raw.githubusercontent.com/torservers/server-config-templates/master/50unattended-upgrades
cp /usr/share/unattended-upgrades/20auto-upgrades /etc/apt/apt.conf.d/20auto-upgrades # enable

Tor

preparation

See also https://www.torproject.org/docs/debian.html.en

# codename of current distribution
DIST=`lsb_release -sc`
# add tor sources
echo "deb     http://deb.torproject.org/torproject.org $DIST main" > /etc/apt/sources.list.d/torproject.list
# only add experimental if you're on tor IRC/familiar with it
echo "deb     http://deb.torproject.org/torproject.org tor-experimental-0.2.8.x-$DIST main" >> /etc/apt/sources.list.d/torproject.list
gpg --keyserver keys.gnupg.net --recv 886DDD89
# if down use eg. keyserver.ubuntu.com
gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add -
sudo apt-get update
sudo apt-get install deb.torproject.org-keyring

setup

apt-get install tor tor-geoipdb
service tor stop
cd /etc/tor
rm *
wget https://raw.githubusercontent.com/torservers/server-config-templates/master/torrc
wget https://raw.githubusercontent.com/torservers/server-config-templates/master/tor-exit-notice.html

# customize
# for multi-process installations, change datadirectory and pidfile (see section further below)

Register at Tor Weather for basic monitoring.

Which Ports Should I Use?

FAQ

Webserver

We have simplified our setup and are currently not using a standalone webserver on our relay servers.

Generate Bandwidth Stats with vnstat(i)

You can generate local graphs and render them as images with vnstati. You could serve these images with a webserver. We don't use this any longer.

apt-get install vnstati
# create empty files in root-owned /var/www and change owner to www-data
cd /var/www
touch vnstat.png vnstat_d.png vnstat_m.png vnstat.xml
chown www-data:www-data vnstat*.*
# set up cron job
crontab -u www-data -e
*/10 * * * * /usr/bin/vnstati -vs -o /var/www/vnstat.png -i eth0 >/dev/null 2>&1 ;fi
*/10 * * * * /usr/bin/vnstati -d -o /var/www/vnstat_d.png -i eth0 >/dev/null 2>&1 ;fi
1 3 * * * /usr/bin/vnstati -m -o /var/www/vnstat_m.png -i eth0 >/dev/null 2>&1 ;fi
1 3 * * * /usr/bin/vnstat --xml > /var/www/vnstat.xml 2>/dev/null ;fi

Local DNS server

apt-get install unbound
vi /etc/resolv.conf # insert top: nameserver 127.0.0.1

Munin Resource Monitoring

Install munin-node and allow remote access from our webserver that runs munin to gather statistics at 194.160.168.61 ( https://www.torservers.net/munin/ ). munin-node is the “client side component”. You might also be interested in the munin "server side component" configuration.

apt-get install -y munin-node
ln -s /usr/share/munin/plugins/netstat /etc/munin/plugins/netstat
rm /etc/munin/plugins/http_loadtime
rm /etc/munin/plugins/ntp_*
rm /etc/munin/plugins/postfix_*
rm /etc/munin/plugins/exim_*
sed "s/allow \\^127\\\.0\\\.0\\\.1\\$/allow ^81\\\.7\\\.13\\\.16$/" -i /etc/munin/munin-node.conf
/etc/init.d/munin-node restart

High Bandwidth Tweaks (>100 mbps?)

You might also be interested in this tor-relay thread regarding high speed relay tweaks: How to Run High Capacity Tor Relays

In general, as with all optimizations: you should only apply those that are necessary for you.

Multiple Tor Processes

Currently, Tor does not scale on multicore CPUs. One Tor process is able to handle around 100mbps of throughput. If your line supports more, you will need to run multiple Tor processes. There is a modified initscript that makes it easy to manage multiple Tor configurations on one machine.

The relevant tor*.cfg settings per relay are (change “0” to “1” etc):

DataDirectory /var/lib/tor/0
PidFile /var/run/tor/tor0.pid
Log notice file /var/log/tor/notices0.log

Note that running more than two tor processes per IP address will result in those other nodes not being used on the network. You'll see the following message in your logs:

[notice] Heartbeat: It seems like we are not in the cached consensus.

(background)

## layout
# ls /etc/tor
tor0.cfg tor1.cfg tor2.cfg tor3.cfg

systemd

wget -O /etc/systemd/system/tor\@.service https://raw.githubusercontent.com/torservers/server-config-templates/master/tor@.service
systemctl enable tor@0
systemctl enable tor@1
systemctl enable tor@2

system v init (pre jessie)

cd /etc/init.d
wget -O tor https://gist.githubusercontent.com/7adietri/9122199/raw/4ed71b894eddbdfb0e241fa06bb583a19f0ccc89/tor
chmod +x tor
# /etc/init.d/tor start    # starts tor 0-3
# /etc/init.d/tor stop     # stops tor 0-3
# /etc/init.d tor reload tor2 tor3
# /etc/init.d/tor stop tor1

</code>

sysctl.conf kernel optimizations

cd /etc
mv sysctl.conf sysctl.conf.dist
wget https://raw.githubusercontent.com/torservers/server-config-templates/master/sysctl.conf
# go through the settings once again! some only useful with large memory and CPU
# better tweaking probably possible; magic involved
sysctl -p

edit rc.local and add sysctl -p

vnstat MaxBandwidth

Set MaxBandwidth to line maximum, eg. for GBit:

sed "s/MaxBandwidth 100/MaxBandwidth 1000/g" -i /etc/vnstat.conf
/etc/init.d/vnstat restart # don't reload; will stop vnstat from updating its db...

TXQueueLen

Might be useful in some cases. Only optimize when you need to!

# remove "exit" from rc.local, then
echo 'ifconfig eth0 txqueuelen 20000' >> /etc/rc.local
# Play with it. For GBit I've found values between 8000 and 16000 to be very useful, but it seems to be hardware dependent

Receive Packet Steering (RPS)

This is highly unlikely to be necessary on modern kernels! If one CPU core is used much more than others (check eg. with htop), enable RPS on kernels >= 2.6.35.

Example /etc/network/interfaces:

auto eth1
iface eth1 inet static
address 123.456.789.01
netmask 255.255.255.255
gateway 012.345.678.90
up echo f > /sys/class/net/eth1/queues/rx-0/rps_cpus

Receive Flow Steering (RFS)

This is highly unlikely to be necessary on modern kernels! Receive Flow Steering (RFS), also introduced with kernel 2.6.35, might help if you run multiple Tor processes, but test first and monitor CPU usage across the CPU cores.

echo 16384 > /sys/class/net/eth0/queues/rx-0/rps_flow_cnt
echo 16384 > /proc/sys/net/core/rps_sock_flow_entries

To make it permanent put it some place useful (can it be put into sysctl.conf?)

AES-NI Crypto Acceleration

Recent Intel CPUs and upcoming AMDs support a native AES crypto acceleration extension called AES-NI. It is well worth enabling and will save a lot of CPU cycles.

Many motherboards ship with AES-NI disabled. You can check if it is enabled:

# cat /proc/cpuinfo | grep aes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 x2apic popcnt aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid

That's all!

Enable AES-NI in Tor

OpenSSL 1.0.1 does not come with an extra module and should directly support AES-NI. You should not use older versions of OpenSSL.

TODO/Extras/Open Discussions

  • tlsdate
  • monitoring: Zabbix?
  • disable webserver referer and user-agent logging, too?
  • selinux/appamor/chroot?
  • firewall: is tarpitting really useful for us?
setup/server.txt · Last modified: 2016/05/25 13:35 by digitalcourage