1 Jul 2011

Headless Testing for Continuous Integration with Ruby and Selenium

A headless system is a computer system or device that has been configured to operate without a monitor (the missing "head"), keyboard and mouse.

This is something you probably want when you're running a continuous integration server and having a bunch of integration tests for your website / application.

Using Firefox on Debian

Web Links

16 Dec 2010

Disable VMware Fusion's vmnet-* daemons on Mac OS X

VMware Fusion initializes some daemons (vmnet-bridge, vmnet-dhcpd, vmnet-netifup, vmnet-natd) at startup. The number of processes depends on the number of network interfaces you have configured. This might slow down your system startup.

Why is it necessary for them to load at system startup rather than application startup?

You need administrator privileges to load kernel extensions (which really makes sense), and these really do need to be kernel extensions (running a virtual machine at acceptable speed requires low-level access), so rather than prompting users for this when they start Fusion (and remember, they might not even know/have the password), we have the kexts get loaded at boot time. Advanced users can certainly disable this and manually start the services when needed.

Source

Here is an example how it could look like:

I don't use VMware Fusion often and want the daemons only to start up if it's necessary. The script to initialize the daemons is located at /Library/Application Support/VMware Fusion/boot.sh and is invoked by the LaunchDaemon script which is located at /Library/LaunchDaemons/com.vmware.launchd.vmware.plist.

  1. deactivate the LaunchDaemon script: "sudo launchctl unload -w /Library/LaunchDaemons/com.vmware.launchd.vmware.plist"
  2. use an apple script to start the boot.sh-script and VMware Fusion:

Here is a short how-to to get the script runnning:

(download)

27 Jun 2009

PHP's pg_connect via pgpool-II vs pgBouncer vs native

In the last few weeks one of our customer had some performance issues with our web application. We discovered a lot of opportunities in our infrastructure that would improve the performance. I’m currently working on an article series about those experiences.

Whatever, if you’re running a high-traffic PHP application with a PostgreSQL database as backend you might be running into problems with slow repsonse times of your pages caused by pg_connect. This was one of the things happens to our application.

To solve this issue you can use a connection pooling software like pgpool-II or pgBouncer. I’m not going to explain what this makes or show how this works in theory. I just give you some statistics fresh from one of our servers to think about it. The webserver is running on Apache 2 + fcgid + PHP (CGI). This explains why the first benchmark results are low and increasing by and by.

Overview

1) PHP file without a database connection

/etc/init.d/apache2 restart ; \ 
  for c in {1..10}; \
  do ab -n 1000 -c 100 http://localhost/phpinfo.php 2>&1 | \
    grep "Requests per second"; \
  done
Restarting web server: apache2 ... waiting ..
Requests per second:     55.04 [#/sec] (mean)
Requests per second:    117.04 [#/sec] (mean)
Requests per second:    108.67 [#/sec] (mean)
Requests per second:     99.30 [#/sec] (mean)
Requests per second:    114.79 [#/sec] (mean)
Requests per second:    155.76 [#/sec] (mean)
Requests per second:    152.06 [#/sec] (mean)
Requests per second:    142.65 [#/sec] (mean)
Requests per second:    154.77 [#/sec] (mean)
Requests per second:    149.60 [#/sec] (mean)

2) PHP file with pg_connect to PostgreSQL

/etc/init.d/apache2 restart ; \
  for c in {1..10}; \
  do ab -n 1000 -c 100 http://localhost/phpinfo2.php 2>&1 | \
    grep "Requests per second"; \
  done
Restarting web server: apache2 ... waiting ..
Requests per second:     8.50 [#/sec] (mean)
Requests per second:    12.47 [#/sec] (mean)
Requests per second:    12.14 [#/sec] (mean)
Requests per second:    12.50 [#/sec] (mean)
Requests per second:    12.50 [#/sec] (mean)
Requests per second:    12.39 [#/sec] (mean)
Requests per second:    12.38 [#/sec] (mean)
Requests per second:    12.34 [#/sec] (mean)
Requests per second:    12.01 [#/sec] (mean)
Requests per second:    11.91 [#/sec] (mean)

3) PHP file with pg_connect via pgpool-II

/etc/init.d/apache2 restart ; \
  for c in {1..10}; \
  do ab -n 1000 -c 100 http://localhost/phpinfo3.php 2>&1 | \
    grep "Requests per second"; \
  done
Restarting web server: apache2 ... waiting ...
Requests per second:     41.43 [#/sec] (mean)
Requests per second:     94.64 [#/sec] (mean)
Requests per second:     80.83 [#/sec] (mean)
Requests per second:     85.47 [#/sec] (mean)
Requests per second:     73.03 [#/sec] (mean)
Requests per second:     87.65 [#/sec] (mean)
Requests per second:     84.18 [#/sec] (mean)
Requests per second:    103.66 [#/sec] (mean)
Requests per second:    105.99 [#/sec] (mean)
Requests per second:    104.72 [#/sec] (mean)

4) PHP file with pg_connect via pgBouncer

/etc/init.d/apache2 restart ; \
  for c in {1..10}; \
  do ab -n 1000 -c 100 http://localhost/phpinfo4.php 2>&1 | \
    grep "Requests per second"; \
  done
Restarting web server: apache2 ... waiting ...
Requests per second:     47.53 [#/sec] (mean)
Requests per second:    106.67 [#/sec] (mean)
Requests per second:     90.49 [#/sec] (mean)
Requests per second:     66.29 [#/sec] (mean)
Requests per second:    102.98 [#/sec] (mean)
Requests per second:    129.06 [#/sec] (mean)
Requests per second:    115.56 [#/sec] (mean)
Requests per second:    132.95 [#/sec] (mean)
Requests per second:    127.06 [#/sec] (mean)
Requests per second:    126.64 [#/sec] (mean)

What are your experiences and which one do you prefer? Why do you prefer pgpool-II over pgBouncer or pgBouncer over pgpool-II?

6 Sep 2006

SELinux and Apache

A problem occured as I tried to start apache on our new RedHat system.

/etc/init.d/httpd start
(13)Permission denied: make_sock: could not bind to
address [::]:10080 no listening sockets available,
shutting down
Unable to open logs

This problem came up after I’ve add “Listen 10080” to the apache configuration file httpd.conf. After a look into /var/log/messages and some research on google I found out that the problem was SELinux. A security system implementation considerably developed by the NSA and RedHat.

grep httpd /var/log/messages
Sep  1 20:21:39 kernel: audit(1157160099.715:11): avc:  denied { name_bind }
for  pid=3058 comm="httpd"  src=10080 scontext=root:system_r:httpd_t
tcontext=system_u:object_r:port_t tclass=tcp_socket

I found some good articles [1], [2] about SELinux, but there was no easy howto for add a security policy that allows apache to use other ports than 80 and 8080.

Now, we will customize Apache for listen on port 10080 and 20080.

If you’ll find no directory /etc/selinux/targeted/src in your RedHat installation, you have to install the selinux sources:

up2date -i selinux-policy-targeted-sources

Afterwards create the file /etc/selinux/targeted/src/policy/domains/misc/local.te and add the line

allow httpd_t port_t:tcp_socket name_bind;

This allows apache to use the name_bind command for every port. A more restricted way is to use /etc/selinux/targeted/src/policy/net_contexts. This way let you approve or disapprove the usage for every port (e.g. portcon tcp 10080 system_u:object_r:http_port_t).

The difference between both is that changes in local.te will survive updates and changes in net_context won’t. [3] If you finished configuration changes you have to compile, install and load your new rules:

cd /etc/selinux/targeted/src/policy
make load

Links

  1. Regel-Praxis by Carsten Grohmann (german)
  2. Writing SE Linux policy HOWTO by Faye Coker (english)
  3. Re: httpd fails to start with latest policy by Stephen Smalley (english)
8 Jun 2006

ImageMagick mit 1und1 (puretec)

1und1 bietet für die Bearbeitung von Bildern die GD-Bibliothek und NetPBM an. ImageMagick selbst wird von Haus aus nicht unterstützt. Auf der Suche nach einer Möglichkeit ImageMagick auch bei 1und1 zu verwenden, bin ich via Google und den Suchwörtern "imagemagick " und "static" auf folgende Lösung gestoßen:

  • ImageMagick static herunterladen
  • entpacken
  • Verzeichnis via FTP/SSH auf den 1und1 Webserver laden
  • Rechte der Dateien auf 755 (''chmod 755'') setzen

Jetzt sollte der entsprechende Pfad bei Gallery oder Typo3 angegeben werden. Fertig.

21 Apr 2006

Filter FTP traffic with tcpdump

This command filters any FTP traffic for ip 10.1.1.22.

tcpdump -i fxp1 -x 'ip host 10.1.1.22 and (port ftp or ftp-data)'
9 Sep 2005

MAC in IP Adresse umwandeln

Irgendwo im Netzwerk steht ein CISCO ATA 186, welches ein SIP Image verpasst bekommen soll. Auf dem Gerät befindet sich lediglich die MAC-Adresse (00:07:...) des Netzwerkinterface, jedoch ohne dazugehörige IP-Adresse. Ein kleiner Trick soll helfen die IP-Adresse herauszufinden:

nmap -sP 10.0.1.0/24
arp | grep "00:07"

Der obige Code, pingt alle IP’s des Class C Netzwerks an, und durchsucht danach die arp-Tabelle nach der entsprechenden MAC-Adresse, und gibt diese inklusive der dazugehörigen IP-Adresse aus. Ihr könnt auch in der von {{nmap}} ausgegebenen Liste nach der MAC-Adresse suchen. Jedoch stehen dort die MAC- und IP-Adresse nicht in einer Zeile, was es unnötig macht, die Liste mit {{grep}} durchzugehen.

9 May 2005

mtop – MySQL Monitoring Tool

Was der Prozessliste sein top, ist dem Query sein mtop. Mittels [mtop][1] lassen sich die aktuell abzuarbeitenden Abfragen beobachten und eventuelle Engpässe z.B. durch schlechte Indizierung, schlechter Queryaufbau, zu große Tabellen, etc. analysieren. [1]: http://mtop.sourceforge.net/ “mtop/mkill – MySQL Monitoring Tools”