Alejandro is right. You need to install mysql-devel package I think. It provides more than just the header files. Check if you also already have the libs e.g. for myself:
/$ rpm -ql mysql-devel /usr/include/mysql /usr/include/mysql/dbug.h /usr/include/mysql/errmsg.h /usr/include/mysql/m_ctype.h /usr/include/mysql/m_string.h /usr/include/mysql/my_config.h /usr/include/mysql/my_global.h /usr/include/mysql/my_list.h /usr/include/mysql/my_net.h /usr/include/mysql/my_no_pthread.h /usr/include/mysql/my_pthread.h /usr/include/mysql/my_sys.h /usr/include/mysql/mysql.h /usr/include/mysql/mysql_com.h /usr/include/mysql/mysql_version.h /usr/include/mysql/mysqld_error.h /usr/include/mysql/raid.h /usr/include/mysql/sslopt-case.h /usr/include/mysql/sslopt-longopts.h /usr/include/mysql/sslopt-usage.h /usr/include/mysql/sslopt-vars.h /usr/lib/mysql /usr/lib/mysql/libdbug.a /usr/lib/mysql/libheap.a /usr/lib/mysql/libmerge.a /usr/lib/mysql/libmyisam.a /usr/lib/mysql/libmyisammrg.a /usr/lib/mysql/libmysqlclient.a /usr/lib/mysql/libmysqlclient.so /usr/lib/mysql/libmysqlclient_r.a /usr/lib/mysql/libmysqlclient_r.so /usr/lib/mysql/libmystrings.a /usr/lib/mysql/libmysys.a /usr/lib/mysql/libnisam.a /$ rpm -q mysql-devel mysql-devel-3.23.58-1.9 James.