Differences
This shows you the differences between two versions of the page.
|
setup:server [2012/06/06 15:24] moritz add vnstat.xml |
setup:server [2013/04/08 06:02] (current) moritz torrc details for multi-tor config |
||
|---|---|---|---|
| Line 14: | Line 14: | ||
| mkdir ~/.ssh | mkdir ~/.ssh | ||
| chmod 700 ~/.ssh | chmod 700 ~/.ssh | ||
| - | vi authorized_keys2 | + | vi authorized_keys |
| </code> | </code> | ||
| - | * paste in your public key; i use 2048 RSA keys | + | * paste in your public key; i use 2048 bit RSA keys |
| * save and try logging in with key | * save and try logging in with key | ||
| Line 23: | Line 23: | ||
| cd /etc/ssh | cd /etc/ssh | ||
| mv sshd_config sshd_config.dist | mv sshd_config sshd_config.dist | ||
| - | wget http://www.torservers.net/misc/config/sshd_config | + | wget https://www.torservers.net/misc/config/sshd_config |
| vi sshd_config | vi sshd_config | ||
| </code> | </code> | ||
| Line 35: | Line 35: | ||
| ==== iptables firewall ==== | ==== iptables firewall ==== | ||
| + | |||
| + | This config by defaults allows world access to the SSH port and Tor ports 80,443. Think about limiting that to an IP (range) you can connect from, or [[http://www.debian-administration.org/articles/268|portknocking]]. Also, this config is optimized for high bandwidth relays: in order to avoid the conntrack module, it allows all UDP in! | ||
| + | |||
| <code> | <code> | ||
| cd /etc | cd /etc | ||
| - | wget http://www.torservers.net/misc/config/iptables.test.rules | + | wget https://www.torservers.net/misc/config/iptables.test.rules |
| chmod 600 iptables.test.rules | chmod 600 iptables.test.rules | ||
| vi iptables.test.rules # update at least SSH port | vi iptables.test.rules # update at least SSH port | ||
| Line 47: | Line 50: | ||
| chmod 600 iptables.rules | chmod 600 iptables.rules | ||
| cd /etc/network/if-pre-up.d/ | cd /etc/network/if-pre-up.d/ | ||
| - | wget http://www.torservers.net/misc/config/iptables | + | wget https://www.torservers.net/misc/config/iptables |
| chmod +x iptables | chmod +x iptables | ||
| </code> | </code> | ||
| - | |||
| - | This config by defaults allows world access to the SSH port and Tor ports 80,443. Think about limiting that to an IP (range) you can connect from, or [[http://www.debian-administration.org/articles/268|portknocking]]. | ||
| ==== a few defaults ==== | ==== a few defaults ==== | ||
| Line 63: | Line 64: | ||
| # disable debian default that pulls in recommended packages: | # disable debian default that pulls in recommended packages: | ||
| cd /etc/apt/apt.conf.d | cd /etc/apt/apt.conf.d | ||
| - | wget http://www.torservers.net/misc/config/06norecommends | + | wget https://www.torservers.net/misc/config/06norecommends |
| aptitude update && aptitude full-upgrade | aptitude update && aptitude full-upgrade | ||
| Line 75: | Line 76: | ||
| cd /etc | cd /etc | ||
| mv aliases aliases.dist | mv aliases aliases.dist | ||
| - | wget http://www.torservers.net/misc/config/aliases | + | wget https://www.torservers.net/misc/config/aliases |
| vi aliases # edit last lime and change to your email address | vi aliases # edit last lime and change to your email address | ||
| newaliases | newaliases | ||
| Line 102: | Line 103: | ||
| # if down use eg. keyserver.ubuntu.com | # if down use eg. keyserver.ubuntu.com | ||
| aptitude update | aptitude update | ||
| + | apt-get install deb.torproject.org-keyring | ||
| </code> | </code> | ||
| Line 168: | Line 170: | ||
| cd /var/www | cd /var/www | ||
| touch vnstat.png vnstat_d.png vnstat_m.png vnstat.xml | touch vnstat.png vnstat_d.png vnstat_m.png vnstat.xml | ||
| - | chown www-data:www-data vnstat*.png | + | chown www-data:www-data vnstat*.* |
| # set up cron job | # set up cron job | ||
| crontab -u www-data -e | crontab -u www-data -e | ||
| Line 176: | Line 178: | ||
| */10 * * * * /usr/bin/vnstati -d -o /var/www/vnstat_d.png -i eth0 >/dev/null 2>&1 ;fi | */10 * * * * /usr/bin/vnstati -d -o /var/www/vnstat_d.png -i eth0 >/dev/null 2>&1 ;fi | ||
| 1 3 * * * /usr/bin/vnstati -m -o /var/www/vnstat_m.png -i eth0 >/dev/null 2>&1 ;fi | 1 3 * * * /usr/bin/vnstati -m -o /var/www/vnstat_m.png -i eth0 >/dev/null 2>&1 ;fi | ||
| - | 1 3 * * * /usr/bin/vnstat --xml > /var/www/vnstat.xml >/dev/null 2>&1 ;fi | + | 1 3 * * * /usr/bin/vnstat --xml > /var/www/vnstat.xml 2>/dev/null ;fi |
| </code> | </code> | ||
| Line 197: | Line 199: | ||
| </code> | </code> | ||
| - | ===== Apparmor profile (unconfirmed!) ===== | + | ====== Munin Resource Monitoring ====== |
| - | * This profile has been tested to work with multiple Tor processes as described here: http://archives.seul.org/or/relays/Sep-2010/msg00004.html | + | Install munin-node and allow remote access from our webserver that runs munin to gather statistics at 194.160.168.61 ( https://www.torservers.net/munin/ ). munin-node is the "client side component". You might also be interested in the [[setup:munin|munin "server side component" configuration]]. |
| - | * The profile should be disabled while Tor is installed for the first time, since different permissions are required for setup and normal operation. | + | |
| - | * We are not using this for Torservers.net. This does not work on our Debian. | + | |
| - | <code># Last Modified: Sat Mar 19 00:38:15 2011 | + | <code> |
| - | # Apparmor profile for Tor with multiple processes on: | + | apt-get install -y munin-node |
| - | # Ubuntu 10.04 | + | ln -s /usr/share/munin/plugins/netstat /etc/munin/plugins/netstat |
| - | + | rm /etc/munin/plugins/http_loadtime | |
| - | #include <tunables/global> | + | rm /etc/munin/plugins/ntp_* |
| - | + | rm /etc/munin/plugins/postfix_* | |
| - | /usr/sbin/tor { | + | rm /etc/munin/plugins/exim_* |
| - | #include <abstractions/base> | + | sed "s/allow \\^127\\\.0\\\.0\\\.1\\$/allow ^194\\\.150\\\.168\\\.61$/" -i /etc/munin/munin-node.conf |
| - | #include <abstractions/nameservice> | + | /etc/init.d/munin-node restart |
| - | + | ||
| - | capability setgid, | + | |
| - | capability setuid, | + | |
| - | + | ||
| - | /etc/passwd mr, | + | |
| - | /etc/tor/* r, | + | |
| - | /usr/share/tor/* r, | + | |
| - | owner /var/lib/tor*/cached* rw, | + | |
| - | owner /var/lib/tor*/cached-descriptors mrw, | + | |
| - | /var/lib/tor*/cached-status/ r, | + | |
| - | owner /var/lib/tor*/cached-status/* rw, | + | |
| - | owner /var/lib/tor*/fingerprint r, | + | |
| - | owner /var/lib/tor*/keys/secret_id_key r, | + | |
| - | owner /var/lib/tor*/keys/secret_onion_key* rw, | + | |
| - | owner /var/lib/tor*/lock rwk, | + | |
| - | owner /var/lib/tor*/state* rw, | + | |
| - | owner /var/log/tor*/* rw, | + | |
| - | /var/run/tor*/tor*.pid rw, | + | |
| - | } | + | |
| </code> | </code> | ||
| ====== High Bandwidth Tweaks (>100 mbps?) ====== | ====== High Bandwidth Tweaks (>100 mbps?) ====== | ||
| Line 236: | Line 217: | ||
| You might also be interested in this or-relay thread regarding high speed relay tweaks: | You might also be interested in this or-relay thread regarding high speed relay tweaks: | ||
| [[http://thread.gmane.org/gmane.network.onion-routing.general/2841|How to Run High Capacity Tor Relays]] | [[http://thread.gmane.org/gmane.network.onion-routing.general/2841|How to Run High Capacity Tor Relays]] | ||
| - | |||
| - | If you're hitting CPU or memory limits, you should look into [[http://moblog.wiredwings.com/archives/20100427/Tor-on-Debian,-self-compiled-for-better-Performance.html|compiling your own Tor]]. | ||
| In general, as with all optimizations: you should only apply those that are necessary for you. | In general, as with all optimizations: you should only apply those that are necessary for you. | ||
| Line 257: | Line 236: | ||
| # /etc/init.d/tor stop tor1</code> | # /etc/init.d/tor stop tor1</code> | ||
| + | The relevant tor*.cfg settings per relay are (change "0" to "1" etc): | ||
| + | |||
| + | <code> | ||
| + | DataDirectory /var/lib/tor/0 | ||
| + | PidFile /var/run/tor/tor0.pid | ||
| + | Log notice file /var/log/tor/notices0.log | ||
| + | </code> | ||
| ===== sysctl.conf ===== | ===== sysctl.conf ===== | ||
| Line 267: | Line 253: | ||
| </code> | </code> | ||
| + | edit rc.local and add sysctl -p | ||
| + | |||
| + | <code> | ||
| + | # An introduction to some of the more interesting, performance related sysctl | ||
| + | net.core.rmem_max = 33554432 | ||
| + | net.core.wmem_max = 33554432 | ||
| + | net.ipv4.tcp_rmem = 4096 87380 33554432 # this seems to be irrelevant for Tor, but is otherwise a good idea for high bandwidth | ||
| + | net.ipv4.tcp_wmem = 4096 65536 33554432 # this seems to be irrelevant for Tor, but is otherwise a good idea for high bandwidth | ||
| + | net.core.rmem_default = 524287 # you can never give the networking enough memory ;-) | ||
| + | net.core.wmem_default = 524287 # you can never give the networking enough memory ;-) | ||
| + | net.core.optmem_max = 524287 # you can never give the networking enough memory ;-) | ||
| + | net.core.netdev_max_backlog = 300000 # backlog size is important for high-bandwidth Tor, as sometimes the kernel is a little | ||
| + | # slower than the rest | ||
| + | net.ipv4.tcp_mem = 33554432 33554432 33554432 # very useful, max amount of tcp memory. Intel recommended setting it to | ||
| + | # 3x10000000, more seems to help more though, and we go around the min,avg,max thing, by just letting it be huge from the beginning. | ||
| + | net.ipv4.tcp_max_orphans = 300000 # set to the same as max_backlog, helps when the system is stressed | ||
| + | net.ipv4.tcp_max_syn_backlog = 300000 # same as netdev_max_backlog | ||
| + | net.ipv4.tcp_fin_timeout = 4 | ||
| + | vm.min_free_kbytes = 65536 # always keep enough free memory to the side for vm | ||
| + | net.ipv4.ip_conntrack_max = 655360 # connection tracking is probably the single most important thing for high bandwidth Tor, | ||
| + | # as we have a huge number of open connections at all times | ||
| + | net.netfilter.nf_conntrack_tcp_timeout_established = 7200 | ||
| + | net.netfilter.nf_conntrack_checksum = 0 | ||
| + | net.netfilter.nf_conntrack_max = 655380 # here applies the same as ip_conntrack_max | ||
| + | net.netfilter.nf_conntrack_tcp_timeout_syn_sent = 15 | ||
| + | net.nf_conntrack_max = 655360 # here applies the same as ip_conntrack_max | ||
| + | net.ipv4.tcp_keepalive_time = 60 | ||
| + | net.ipv4.tcp_keepalive_intvl = 10 | ||
| + | net.ipv4.tcp_keepalive_probes = 3 | ||
| + | net.ipv4.ip_local_port_range = 1025 65530 | ||
| + | net.core.somaxconn = 30720 # 20480 should be enough for every situation, but its a good one for playing around | ||
| + | net.ipv4.tcp_max_tw_buckets = 2000000 # you should keep a few of those around as a bunch of connections will always be in | ||
| + | # this state. 2000000 is probably a little high, but won't do any harm, either. | ||
| + | net.ipv4.tcp_timestamps = 0 | ||
| + | </code> | ||
| ===== vnstat MaxBandwidth ===== | ===== vnstat MaxBandwidth ===== | ||
| Line 274: | Line 295: | ||
| /etc/init.d/vnstat restart # don't reload; will stop vnstat from updating its db...</code> | /etc/init.d/vnstat restart # don't reload; will stop vnstat from updating its db...</code> | ||
| - | ===== TXQueueLen, MTU "Jumbo Frames" ===== | + | ===== TXQueueLen ===== |
| Might be useful in some cases. Only optimize when you need to! | Might be useful in some cases. Only optimize when you need to! | ||
| Line 281: | Line 302: | ||
| # remove "exit" from rc.local, then | # remove "exit" from rc.local, then | ||
| echo 'ifconfig eth0 txqueuelen 20000' >> /etc/rc.local | echo 'ifconfig eth0 txqueuelen 20000' >> /etc/rc.local | ||
| - | ifconfig eth0 mtu 9000 | + | # Play with it. For GBit I've found values between 8000 and 16000 to be very useful, but it seems to be hardware dependent |
| - | # to make permanent, add "mtu 9000" to /etc/network/interfaces | + | |
| </code> | </code> | ||
| Line 347: | Line 367: | ||
| ===== check openssl AES-NI support ===== | ===== check openssl AES-NI support ===== | ||
| - | OpenSSL does not come with AES-NI support by default. Fortunately, Ubuntu ships with a patched version. | + | OpenSSL <1.01 does not come with AES-NI support by default. Fortunately, Ubuntu ships with a patched version. |
| <code> | <code> | ||
| Line 355: | Line 375: | ||
| </code> | </code> | ||
| + | OpenSSL 1.0.1 does not need an extra module and should directly support AES-NI: | ||
| + | |||
| + | <code> | ||
| + | # openssl engine | ||
| + | (rsax) RSAX engine support | ||
| + | (dynamic) Dynamic engine loading support | ||
| + | </code> | ||
| ===== Benchmark openSSL ===== | ===== Benchmark openSSL ===== | ||
| Line 375: | Line 402: | ||
| ===== Enable AES-NI in Tor ===== | ===== Enable AES-NI in Tor ===== | ||
| - | Edit torrc and restart Tor (reloading won't do). | + | OpenSSL 1.0.1 does not come with an extra module and should directly support AES-NI. |
| + | |||
| + | For OpenSSL <1.0.1, edit torrc and restart Tor (reloading won't do) to use the aesni module. | ||
| <code> | <code> | ||
| Line 384: | Line 413: | ||
| ====== TODO/Extras/Open Discussions ====== | ====== TODO/Extras/Open Discussions ====== | ||
| - | * TODO: public traffic/other stats. i didn't like cacti very much... | ||
| * monitoring: Zabbix? | * monitoring: Zabbix? | ||
| * disable webserver referer and user-agent logging, too? | * disable webserver referer and user-agent logging, too? | ||
