I would like to request you to mention your need more clear so anybody can help you.

Security of any application is very vast area and how to execute security testing is purely depends on so many factors. You will get common check list for web based security testing on below link.

http://software-testing-qa-guide.blogspot.com/

1. Type of application (Web, Window, etc)
2. Requirement
3. Need to define scope
4. nature of application
5. Type of Transactions

If we talk about web security then there are various methods are there to execute security testing. Difficult to list all but few of are:

- SQL injection
- Cross site scripting
- URL jumping \ Manipulation
- Session hijacking
- Cookie stealing
- buffer overflow, Etc

TO execute above testing , you have to understand basic of tricks and need to prepare test data and environment for that in case of MANUAL Testing.

If we will talk for Automated tools which can help you to execute above testing there we have commonly two types of tools are there:

A) Free Tool
- SQL ME
- XSS ME
- OWASP (Good tool as per my option), etc

B) Paid Tool
- HP WebInspect
- Acunetix Web Vulnerability Scanner, etc

For more information refer following site: http://www.softwareqatest.com/qatweb1.html

Let me know if you have any query for specific area.