|
Alec Matusis
matusis at yahoo.com
Tue Aug 5 04:25:03 EDT 2008
One more interesting piece of evidence: when I connect to the DB on the same physical machine were my web scripts are, I can reproduce this only when I connect to 127.0.0.1. When I connect to localhost, I do not get the errors at all. This is because http://dev.mysql.com/doc/refman/5.0/en/connecting.html “For connections to localhost, MySQL programs attempt to connect to the local server by using a Unix socket file. This occurs even if a --port or -P option is given to specify a port number. To ensure that the client makes a TCP/IP connection to the local server, use --host or -h to specify a hostname value of 127.0.0.1” This last thing makes me think it’s a networking problem as you suggest ? From: Buck Golemon [mailto:workitharder at gmail.com] Sent: Sunday, August 03, 2008 9:24 PM To: Alec Matusis Subject: Re: [mod_python] intermittent bug with MySQLdb Have you considered that this may be just a general problem of networking / MySQL? That's my conclusion. I've gotten similar errors in my high-concurrency applications (non mod-python) and to solve the problem, wrote some code to re-try queryies on that error (among others) and wrapped it around all my connections. As a counter-argument (if you want one), try doing a similar stress test using a completely different client (maybe perl?) and show that the error does not occur under similar or higher loads. --Buck 2008/8/1 Alec Matusis <matusis at yahoo.com> I was stress- testing MySQL replication on my development setup and run into a particularly nasty case of a hard-to-reproduced bug that I have encountered earlier. Under high concurrency, I sometimes get build/bdist.linux-i686/egg/MySQLdb/connections.py: OperationalError: (2003, "Can't connect to MySQL server on ''10.18.0.2 (4)") Here (4) stangs for: #perror 4 OS error code 4: Interrupted system call I am using apache 2.2.6 with worker MPM , python 2.4.4 and mod_python 3.3.1. 10.18.0.2 is a remote DB test server. I upgraded MySQLdb adapter from 1.2.0 to 1.2.2 and got the same error with both versions. The adapters are compiled with mysqlclient_r , which is supposed to be thread-safe. Then, I pointed MySQL connections to localhost, which runs an identical copy of the same database. To get errors on localhost, I had to increase the request rate by about 10x (no errors at all otherwise), and I got OperationalError: (2003, "Can't connect to MySQL server on '127.0.0.1' (4)") 127.0.0.1 is my development machine, and I have never seen these errors before, until I ran this stress test today, bringing the machine to load average 8.0. 127.0.0.1 and 10.18.0.2 are two different machines, with different kernels and even different MySQL versions, so it is the client machine that is a problem. I have written a single-threaded stand-alone python loop, that simply connects to the DB on the remote server 10.18.0.2 and executes "SELECT 1". I have run several of these loops concurrently each doing about 10000 iterations, and they never gave a connection error. MySQLdb README says: MySQLdb is an interface to the popular MySQL_ database server for Python. The design goals are - Thread-safety - Thread-friendliness (threads will not block each other) So my guess is that this is a thread-safety problem having to do with mod_python. I now recall that I have seen the problem earlier: I have several identically configured production web servers connected to a single DB server. I have seen these errors infrequently on 32bit webservers, but extremely rarely or almost never on 64bit webservers. I since phased out 32bit web servers to serve static files only, so I do not have this data anymore. _______________________________________________ Mod_python mailing list Mod_python at modpython.org http://mailman.modpython.org/mailman/listinfo/mod_python -- Buck Golemon AIM: bukzor Cell: (480) 272-1153
|