First part we will install web server base as LAMP, LAMP stack is a group of open source software used to get web servers up and running. The acronym stands for Linux, Apache, MySQL, and PHP. Since the server is already running CentOS, the linux part is taken care of. Here is how to install the rest.

1. Preparation

Fresh server (VPS) with CentOS (5-6.4)
Keep your server updated
yum update
Add extra repositories
rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm

2. Install Apache (httpd)

Apache is a free open source software which runs over 50% of the world's web servers.
Use this command to install httpd
yum install httpd
Once it installs, you can start apache running on your VPS:
service httpd start
Make running on startup
chkconfig httpd on

3. Install MySQL

MySQL is a powerful database management system used for organizing and retrieving data on a virtual server
To install MySQL, open terminal and type in these commands:
yum install mysql-server
Start service
service mysqld start
Config on startup
chkconfig mysqld on
During the installation, MySQL will ask you for your permission twice. After you say Yes to both, MySQL will install.
Once it is done installing, you can set a root MySQL password:
/usr/bin/mysql_secure_installation

4. Install PHP

To install PHP on your virtual private server, open terminal and type in this command:
yum install php php-mysql php-gd php-mcrypt

5. Test server

To set this up, first create a new file: (use nano or vi as you know)
nano /var/www/html/info.php
Add in the following line:
<?php
phpinfo();
?>
Then Save and Exit.
Restart apache so that all of the changes take effect on your virtual server:
service httpd restart
Finish up by visiting your php info page (make sure you replace the example ip address with your correct one): http://12.34.56.789/info.php