UdmSearch performance issues
============================


Post-indexing optimization
--------------------------

This article was supplied by Randy Winch <gumby@cafes.net>


I have some performance numbers that some of you might find interesting.
I'm using RH 6.2 with the 2.2.14-6.1.1 kernel update (allows files
larger than 2 gig) and mysql 2.23.18-alpha.  I have just indexed most 
of our site using udmsearch 3.0.18:


          UdmSearch statistics

    Status    Expired      Total
   -----------------------------
       200     821178    2052579 OK
       301        797      29891 Moved Permanently
       302          3          3 Moved Temporarily
       304          0          7 Not Modified
       400          0         99 Bad Request
       403          0          7 Forbidden
       404      30690     100115 Not found
       500          0          1 Internal Server Error
       503          0          1 Service Unavailable
   -----------------------------
     Total     852668    2182703


Not a small database :-)


I optimize the data by dumping it into a file using "SELECT * INTO OUTFILE", 
sort it using the system sort routine into word (CRC) order and then reloading 
it into the database using the procedure described in the mysql online manual:

http://www.mysql.com/documentation/mysql/commented/manual.php?section=Insert_speed

The performance is wonderful. My favorite test is searching for "John Smith".
The optimized database version takes about 13 seconds. The raw version takes 
about 73 seconds.

Search results: john : 620241 smith : 177096
Displaying documents 1-20 of total 128656 found

Randy Winch


