Friday, February 7, 2014

Data Driven Testing in soapUI open source

Preparing the test environment 


  • Create a folder where SOAP data driven testing will occur, e.g. C:\SOAPUIRulesTest 
  • In the project folder (e.g. DTTest) is where the SOAPUI project will be stored 
  • In the project folder, create a result folder for each instance of the testing you want to run, e.g. output
  • In the date/time stamped test case folder is where the following will reside 
    • input CSV file 
    • results XML files from SOAPUI 
    • results spreadsheet Preparing the input CSV file 
    • Create a CSV file into the test case folder, e.g. input.csv 
    • Format the CSV file as follows: 
    • First row is the name of tags which the values will be replaced with 
    • Row 2 onward are the tag values 
    • The last column is the expected result field - this is used to create an assertion in SOAPUI against which to test

soapUI Project 


  • Open soapUI and create a new project using the WSDL path for the Rule Service - keep the name short as long names may cause path issues down the line 
  • Create the test suite as part of the project creation or afterwards

  • Save the project into theproject folder created earlier (not the date/time stamped folder) Test Suite
  • Create a new test suite

Test Case 


  • Create a new test case - keep the name short line "TestCase1" - this name will be used later


  • Select the test case and select options


  • Make sure you unselect "Abort on Error" Test Request  Add Test Request and select "Create Optional Elements"


  • You should now see the SOAP XML structure, e.g.


  • For all "?" items in tag values, replace with substitution strings like "${#TestCase#TAGNAME}"
Setup Script 

  • In the test case open the "Setup Script" tab and paste the following code inside 

//Setup script: 
 //def directory = "C:/SOAPUIRulesTest/getClientRiskAppetiteClassification" def directory = "c:\{path}/result" 
 //Put readers/writers into context using the context variable so it can be used between test components 

context.fileReader = new BufferedReader(new FileReader(directory+"/"+"input.csv")) 
context.fileWriter = new File(directory+"/"+"output.csv") 
context.fileWriter.delete() context.resultscount = 0 

 //Read in the first line of the data file firstLine = context.fileReader.readLine() 

//Split the first line into a string array and assign the array elements to the field name array 

String[] strArray = firstLine.split(",") 
context.fieldNameArray = strArray 

 //Read in the second line of the data file 
secondLine = context.fileReader.readLine() 
String[] propData = secondLine.split(",")
 //Output the headings 

p=0 
for (i=0; i<context.fieldNameArray.size(); i++) 

{
       context.fileWriter.append(context.fieldNameArray[i]+",")
}
context.fileWriter.append("TestResult"+"\r\n")

//set the properties for the first record
p=0
for (i=0; i<context.fieldNameArray.size(); i++)
{
                testCase.setPropertyValue(context.fieldNameArray[i],propData[p++].trim()) //e.g. "uniqueId" = propData[0])
} 
p=0
for (i=0; i<context.fieldNameArray.size()-1; i++)
{
                context.fileWriter.append(propData[i]+",")
}

//Rename request test steps for readability in the log; append the element name to the test step names
testCase.getTestStepAt(0).setName("TestRequest_" + propData[0]+"_"+propData[propData.size()-1])



  • Change the file path to where the CSV file is stored (in the date/time stamped folder)
  • Adjust all the "setPropertyValue" statements to match the xml tags mentioned in the Test Request substitutions earlier

Groovy Script (ReadNextLine)
  • Now add a Groovy script called "ReadNextLine"

  •  Now paste the following code into the window for the script: 


//Setup script: 
 //def directory = "C:/SOAPUIRulesTest/getClientRiskAppetiteClassification" 
def directory = "C://result" 



 //Put readers/writers into context using the context variable so it can be used between test components context.fileReader = new BufferedReader(new FileReader(directory+"/"+"input.csv")) 
context.fileWriter = new File(directory+"/"+"output.csv") 
context.fileWriter.delete() 
 context.resultscount = 0 

 //Read in the first line of the data file 
firstLine = context.fileReader.readLine() 
//Split the first line into a string array and assign the array elements to the field name array 
String[] strArray = firstLine.split(",") 
context.fieldNameArray = strArray 

 //Read in the second line of the data file 
secondLine = context.fileReader.readLine() 
String[] propData = secondLine.split(",") 

//Output the headings 
p=0 
for (i=0; i<context.fieldNameArray.size(); i++) 
{
       context.fileWriter.append(context.fieldNameArray[i]+",")
}
context.fileWriter.append("TestResult"+"\r\n")

//set the properties for the first record
p=0
for (i=0; i<context.fieldNameArray.size(); i++)
{
testCase.setPropertyValue(context.fieldNameArray[i],propData[p++].trim()) //e.g. "uniqueId" = propData[0])
}

p=0
for (i=0; i<context.fieldNameArray.size()-1; i++)
{
                context.fileWriter.append(propData[i]+",")
}

//Rename request test steps for readability in the log; append the element name to the test step names
testCase.getTestStepAt(0).setName("TestRequest_" + propData[0]+"_"+propData[propData.size()-1])


Adjust all the "setPropertyValue" statements to match the xml tags mentioned in the Test Request substitutions earlier

Assertion
  • Click on the Assertion tab for the Test Request


Add details as:


  • This sets the assertion to match the last column of the CSV file
  • Run the test case to make sure all tests now pass




You can also run the test case in test runner and capture every request-response

Monday, October 14, 2013

Changes to Testing Methods for SOA


The adoption of SOA for agility, reducing lifetime cost of an application or accelerating time to market for new business features, does require a change in your current testing strategy. These changes will reduce the amount of time it takes to test, enabling you to move faster through test cycles, and will require new activities (e.g., automated continuous regression testing). Complete testing of all possible paths of a program, an application, or system has become impractical if not cost-prohibitive for organizations (and has been for some time). There are just too many paths through a program to test; Yet, defects exist, and organizations must do intelligent testing and full life cycle testing such that defects are identified. For many test teams or test groups, deploying business applications into production with defects is way of life, where defects are now categorized and workaround techniques are communicated in lieu of fixing the defects. It does not have to be this way.
SOA provides an opportunity to improve testing when using services. The use of services promotes black-box testing, where functional tests can focus on valid inputs and outputs. The service contract defines valid inputs and expected outputs of a service, which promotes the use of black-box testing and tools to test the service. Blackbox testing eliminates the need to perform complete testing of all possible paths of a program, application, or system. In most environments, testing of a changed or newly deployed application requires retesting of components in the new system and their connections to downstream systems. This testing cycle is fraught with errors. Often, it must be performed in a linear fashion, and calendar time is slowly eaten away, causing a decision to either delay deployment, turn off features, or simply to live with defects in a production system (where operator workarounds replace functioning software). SOA fixes this issue because a deployed and working service does not need to be retested or included in future test cycles when reusing the service and deploying a new applications that uses the production-deployed service.



Figure shows the difference of scope with business applications before and after adopting SOA. In Figure, the scope of what needs to be tested is larger before SOA. The scope of the test cycle is less with the use of services when a service life cycle is introduced, where each service goes through a testing cycle in the same manner as we treat applications. Just like applications, services do not have to be retested after they are deployed into production. Services can be certified as satisfying their contract specifications. This certification provides consumer’s confidence that the service works as designed and results in less overall testing when applications are structured using services.
      With SOA adoption, where services are the structuring element of the application, black-box testing becomes the norm. This avoids the retesting of services already deployed in production to determine whether the new system, application, or service has a bug, because services working in production continue to accept the proper inputs and deliver the correct outputs according to the service contract. SOA testing is facilitated when automated continuous regression testing is used. Regression testing facilitates the rapid testing of services as black boxes to ensure the service contracts and services work as planned. Test drivers also prove useful for enabling the consumer to test the provider service in a controlled environment and without always having the platform of the service provider available. Service virtualization testing products enable test teams to mimic the functionality of a service based on its contract design before the implementation is developed, which further improves the quality of the service. 


Friday, September 27, 2013

Database Testing using HP-UFT and CA-LISA

CA-iTKO LISA - Database Testing Tool

   It is considered always better to test the data and perform database validation. Due to the defects in the store procedures, which corrupt the data, truncate or dump the data in incorrect column of the table. When we try to view the same from GUI, it fails as the information is incorrect or sometime even not available. Many time the test team had to go to the database level to perform the database validation. This requires the tester to have knowledge and understanding of database schema and how to create and execute SQL queries. If you are doing database validation you must be aware of what kind of pain it is to create and maintain those scripts. Each time the values are to be pulled from screens, XML and manually passed to SQL queries. For few records its feasible, but imagine if you have to validate thousands of records and that to on regular basis. Recently I had an opportunity to check the features of HP UFT. UFT can be utilized to perform the database validation but the cost involved is very high, but if you already have UFT or LISA below is the brief of how their approach are to achieve this.
 HP-UFT:
Pre-requisite for UFT is to install database client. Eg. If you are using Oracle as your database, first you will have to install Oracle Client on the system where you UFT is available. This is required to connect to the database and later execute SQL queries.
Step 1. Start UFT and create sample test case. On the canvas, follow step are required to connect to database
Step 2. Database -> Open Connection
Step 3. Database -> Select Data
Step 4. Database -> Execute Command
These are the steps which are required each time. For validation there are no in-build assertions which can be utilized. You will have to rely on checkpoint and will have to parameterize the expected value.
Step 5. To pull the records from database follow steps 1 & 2. Use write to file step to dump the records to file. Here all the records are written in tags eg. '<'ROW'>''<'IDENTIFY_DATA'>''' To avoid this you will have to do some scripting. If you wish to perform validation on the values from this file again it’s difficult as the file is not an XML file and the data is not arranged properly
Problem:
Difficult to add database validation.
Mapping of external data is point to point and takes effort to update / enhance if required
When data is pulled it is in incorrect XML format which is difficult to validate or use in other test cases
 CA-LISA
Pre-requisite for LISA is to add the database .jar file in LISA lib/hot deploy folder.
Step 1. In LISA add JDBC step.
        a. Select / Add JDBC driver from the dropdown. This list is available because you have added .jar file of the database
        b. Enter the connection string in the text box
        c. Enter User name & password
        d. Enter the SQL query which you want to execute
        e. You are now ready to add filters and assertions on the result set.
To pull the records from database simple add filter 'Write Properties to File' to the same step and all the values will be available in the mention file.
If you compare both the approach of the tools you will notice that in LISA you can achieve the same in just 1 step.
Problem: Only problem observed is to get the correct .jar file for database connectivity.
    Overall compared to above mentioned two tool, database validation using LISA seems to be far easy. The same can be used as test data for other test cases. As LISA framework is built on properties it is very easy to use the captures values through properties in complete LISA project.

You are welcome to provide your thoughts on the above.

Monday, July 30, 2012

Database Testing

Why test the database?

The database persists data that your application (and organization) depends on. The data thus persisted is most often of mission-critical nature and a key asset for the organization. Also, many of today's data-enabled applications implement a fair amount of their functionality and business logic in the database itself. For an enterprise class application, the data in the database would be accessed and updated (insert/modify/delete) simultaneously by a large number of users (think thousands to millions depending on the scale of your application's usage).

The above statements highlight a few areas where database testing is needed. One, to validate the quality of data being persisted. Two, if we plan to test the application code, it is imperative that we also test the code in the database which implements the business functionality and three, we should plan for non-functional database testing to support the usage of the database in a real-world deployment scenario.

Let us briefly look at the above mentioned database test areas.

1. Data quality testing

Testing the quality of the data may be approached in three ways - data validity testing, data integrity testing and data format testing.

a) Data validity testing - is done to verify the validity of the data that is stored in the database. When data is entered via the front end application, check if the data is correctly updated in the back-end database. Apart from the positive checks, look for other behavior such as data truncation, verify how null/empty field values are handled, verify how special characters or code snippets are handled in the database. Check that the right columns in the right tables are being updated. Data validity testing normally involves use of SQL queries to validate the data.

b) Data integrity testing - involves testing referential integrity and application of constraints (foreign/primary key). When a data field is subject to modification (insert, update or delete), the database should be verified for appropriate changes to related entities such as primary key/foreign key relationships and that referential integrity of the data is maintained

c) Data format testing - involves verifying the size and type of fields that store data in the database with those that accept data in the application. This can help identify mismatches between the type or size of data that is accepted by the front-end vs what the database can store. Example: the application may accept text data but try to store in a numeric or date field in the database or else the application may accept data of greater length than the max length for the corresponding field in the database. This may not throw errors during routine application usage but may store incorrect or erroneous data in the database which could have repercussions elsewhere or at a later stage
   
2. Database code testing - involves testing the code in the database which implements business logic and functionality. Examples of such code include, stored procedures, views (read-only/updateable) and event driven items such as triggers. Each stored procedure is tested distinctly for its functionality. When a stored procedure implements multiple functions, each function is tested separately. Stored procedure testing would look at testing the arguments that are passed to the stored procedure in terms of the number, type and order of arguments plus the return value. Both positive and negative tests can be devised to test stored procedures. Views both read only and updateable are tested either as stored queries that dynamically retrieve values from the database and/or allow updates to the database. In case where updates are allowed, data validity and integrity testing is done. Event driven items are tested by verifying the events that could trigger actions and the actions themselves for functional correctness

3. Non-functional database testing

In most real-world mission critical deployments, to ensure database scalability, security, availability and recover-ability with minimal/no-loss of data, it is important to test the following non-functional areas

a) Database performance testing (load/stress/longevity/scalability)
b) Database security testing
c) Database replication testing
d) Database fail-over testing
e) Database recovery testing

Tuesday, January 31, 2012

SOA Implementation Methodology: What to look for?

SOA Implementation Methodology: What to look for?
Organizations planning to implement SOA should implementation methodologies which covers end-to-end implementation of the SOA roadmap. It should help organization to select the appropriate approach for SAO implementation and set up the IT goal. The methodology should provide visibility of challenges, risks and ROI. 
It should be classified into phases: 
• Formalizing the roadmap, domain model and goal model 
• Providing the component, message, service and information specifications 
• Providing the final implementation based upon standards 

SOA Governance framework: 
Enterprises SOA governance framework should be based upon Open-Source platform. Framework refers to the standards and policies that govern the design, build and implementation of an SOA solution and the policies that must be enforces during runtime. Organizations should also indentify complete testing framework for Unit Testing, Functional Testing, Integrated Testing and Process-level Testing to ensure a high quality of service.

iTKO LISA Vs QTP

Feature iTKO LISA Quick Test Professional
Cross browser testing IE, Firefox, Safari, etc IE and Firefox possible
Supported Technologies LISA supports DTHML/JavaScript, AJAX, Java Applet, Swing, Flash/Flex, Microsoft ActiveX, Oracle Apps Separate license required
SOA Test validation LISA has advantage of connecting middle layer interfaces to test SOA Architecture Scripting required to connect interfaces. Limited to Request and response validations
Validations or Check points Assertions available with support for more than 60 middle layer technologies Check points possible for supported technologies
User friendly Interface Test flow is represented diagrammatically and easy to understand. Test Flow is complex
Control test flow Easy to rearrange test flow as it is Drag & drop Rearranging the Test flow is complex and there is dependency on QTP resource
Implementing Validation Codeless filters and assertions Scripting by use of “if conditions”
Synchronization Easy to create Sync point at step level as well globally. Scripting required
Creating Functions Easy to convert test cases in to sub process. Function creation is possible using scripting
Database Validation Codeless effort for DB connectivity , fetching data and comparing with application Scripting required to validate data base , there are some check points through which we can connect data base without code but still required coding to validate application
QC Integration Available Available
Load & Performance Testing Available Not Possible
Stubs and Drivers LISA Virtualization available to overcome testing constraint Not Possible
Object Identification (GUI) Customization is difficult to implement. Record and Playback option available Customization possible like Descriptive programming
Coding according to Functionality to validate Data on App(GUI & Web Services) Required Java coding knowledge Possible with VB scripting in QTP
Customized report Supports Excel, CSV and pdf without single line of scripting We can customize reports into various formats with the help of scripting
Web Services Testing Easier way without scripting Scripting required or else can use Web service add-in still need some scripting for validations
Automation using Design documents Not Possible Possible

Monday, May 9, 2011

LISA – No Code Testing Tool

Hello Friend! If you are reading this blog that means you are already aware of the testing tool iTKO LISA. You may have heard many times that it is considered to be a no-code testing tool. I would like to share case-study which will prove that really LISA is a no-code tool.

Overview
The project was developed for a major US insurance company. It is in the process of re-architecting its complex insurance policy system to meet broader insurance market needs, by replacing many legacy data transfer and business process technologies with an ESB and SOA-based architecture, the firm is implementing systems that can flexibly scale and deliver new functionality faster to meet the needs of a larger insurance marketplace.

The Challenge
Policy Administration is the heart and soul of any insurance company. Due to the vast complexity of the system and the agile development cycles it was very difficult for the QC team to perform testing. Whenever any changes are implemented the test team had to perform smoke test before starting with the functional and regression cycles. Smoke test used to take couple of day to complete as different downstream were involved and the QC team has to validate the processed policies. Few policies were identified for the smoke test and were processed. All the processed policies were zip in a folder which used to have CCYYDDMMHHMMSS.policy.zip naming convention. The QC had to indentify the zip folder which was processed in a particular run and validate all the policies are available. Once the policies are processed a copy of the policy.zip folder used to be maintained at the archive location. The QC had to copy the folder from archive location to the local landing zone which was part of the identified frame work and extract the policy files and confirm all the policies are processed successfully. Match each policy file is available in the processed .zip folder from the archive location with the policies they have processed, execute SQL queries to perform audit database validation which involved more than 50 validations such as status log, verify the column values as per business rules, processed dates. The status log can have Error or duplicate as the status of the file if it was reprocessed or some error occurred while processing the policy files. If the status is error the detailed description of the error had to be captured which can be shared to the development team. If the status is duplicate, the previous date of the policy when it was processed had to be captured. If the policies are processed successfully the details of the downstream for the policy had to be captured. This had to be done for each and every policy which was processed and for every release. It took QC team more than 2 days to complete the smoke test and later start with the actual testing, so there was a requirement to automate this process so as to reduce the duration of smoke test so that the QC team can utilize the time to concentrate on function and regression test.

The Solution
iTKO LISA product suite offered the broad ability to test, validate most complex elements and as the tool was identified for functional and regression testing, a script was created for smoke testing. Different DOS commands were used to capture the .zip folder name, copy the folder to local system, extract the file and compare the files with the files from archive location. Different available filters and assertions were used to perform this. JDBC steps were been used to perform Audit database validation. A user customized report was generated in .CSV which used to provide the required details for each processed policy.

The Result
With the help of the scripts QC team was able to perform smoke test in less than one hour for each and every policy with customized report which would provide the required details which can help the development team to identify the issues and work on them and reduce the release cycles. QC team was able to perform the smoke test and was able to concentrate on functional and regression tests. The script was used for each release to perform smoke test. The scripts were created using all the default features available in the tool and not a single line of java scripting was used.