The latest version of the README.win32.txt for DBD.Oracle has a faily detailed tutorial on how to compile a DBD modual on windows.
http://svn.perl.org/modules/dbd-oracle/trunk/README.win32.txt It is for DBD:Oracle but is should work much the same for DBI::Mysql