|
Sub Category
|
Assessment
Question
|
Process
|
Agile (Concept -> Production
Timeframe)
|
Do
you follow agile?
|
If not Agile, then what methodology
does the project follow?
|
Sprint
duration?
|
Production Release cycle?
|
Project
Management/Tracking Tool used ? Eg: Rally/Jira?
|
Number of Builds in a Release
|
What is the build frequency? (Ex:
Daily/Weekly..)
|
What
is the build failure percentage?
|
Who does the build deployment?
|
Is
build automated?If automated which tool is used?
|
If build fails, who does the
rollback
|
Cloud/Legacy?
|
Is
the cloud or Legacy? (server/DB)
|
Is there a plan of moving to cloud
if it is legacy?
|
Team Size (Full Team - Dev/OPS and QA)
|
What
is the team size?
|
How many QA members?
|
Is
Dev or Ops internal or manage by contractors
|
Prod/Lower Env Downtime (Average)
|
How often the environment go down?
|
What
is the reason for downtime?
|
Is it self recovery?
|
QA Team Skills
|
How
many in the team contribute to automation development?
|
Does the QA team perform any
Unit/Integration testing?
|
Coding
|
Automated version control
|
Which
tool is used for Version Control?
|
Static Code Analysis
|
Is static Code analysis is done in
the application?
|
Which
tool is used for Static code analysis?
|
How often analysis is done?
|
Code Coverage
|
Is
code coverage being done in the application?
|
|
Which tool is used for Code
coverage?
|
Code Review
|
Is
code review being done ?
|
|
Is the process automated/manual?
|
Continuous Integration
|
CI/CD Pipeline Established using a
tool
|
Is
CI/CD done?
|
Which tool is used?
|
Is
the pipeline set up for all environments including Prod (During prod ..)
|
Automated Build
|
Is Build is automated?
|
Which
tool is used?
|
Automated Unit Testing
|
Is unit testing Automated?
|
Which
tool is used to automate?
|
Is the unit test is integrated with
CI/CD?
|
Code Quality Analysis
|
Is
Code Quality being done for the application?
|
Which tool is used for analysis?
|
Who
is the responsibility for the activity?
|
Artifact Repository
|
Which tool is used as Artifact
Repository?
|
Who
is the owner for this?
|
Automated Deployment
|
Is the deployment automated?
|
Which
tool is used for deployment?
|
What is the deployment failure
rate?
|
In
case of failure, Who is the responsible for the roll back?
|
Continuous Testing
|
Automated Build verification testing
|
Is build verification testing is
done?
|
Is
build verification is automated?
|
Is it integrated with CI?
|
InSprint Automation
|
Is
automation applicable in the application?
|
Is InSprint automation done?
|
What
is the average % of InSprint automation?
|
Is insprint automation integrated
to CI?
|
Automated Regression Testing
|
Is
automation possible?
|
Is Regression test suite is
automated?
|
What
is the regression automation %?
|
What is the automation Utilization?
|
What
is the one click percentage/first pass rate
|
What is the automation defect efficiency
|
What
is the effort spent in maintenance of scripts?
|
Automated Performance Testing
|
Is performance testing done?
|
Which
tool is used for Performance automation?
|
Who is responsible for performance
automation?
|
How
often the performance testing done?
|
Is performance testing is
integrated with CI?
|
Service Virtualization
|
Is
service virtualization done in the application?
|
Environment Automation
|
DB Deployment Automated
|
How does DB deployment happen?
|
Is
there any tool that deploys all the DB Scripts at a time?
|
Env. Definition & Planning
|
How many environments do you have?
|
Environment
setup is it a manual or automated process?
|
How many servers are maintained for
each environment?
|
Who
is involved in planning the environments setup?
|
Env. Booking & Scheduling
|
How many virtual Machines are allotted to each environment?
|
Is
there any planned schedule before the deployment?
|
How do you maintain the Virtual
servers without overlapping between environments?
|
Env. Configuration
|
Is
there any automated process followed to configure the environments?
|
Who is responsible for handling the
configuration issues?
|
What
are the methods involved in configuring servers?
|
Env. Refresh & Recycle
|
Who is responsible for
refresh/recycle the servers?
|
Is
there any automated tool to refresh & recycle the servers?
|
Test Data & Virtual
Services Provisioning
|
Is there any test data dependencies
observed?
|
How
do you rectify the test data issues?
|
How do you provision virtual serveries?
|
Who
is responsible for provisioning the virtual services?
|
Env. Servicing
|
Is there any automated approach
adapted to fix the environment issue?
|
How
do you maintain environments?
|
Continuous Release
|
Release Planning
|
Do you have a sprint grooming and
planning meeting?
|
Is
your release calendar finalized in the beginning of the year
|
Are the releases launched in prod
as planned?
|
Release Automation
|
Are builds
promoted to prod automatically using continuous delivery?
|
If not, then how are the builds
promoted to production?
|
Release monitoring
|
Is
there any monitoring tool that triggers alerts after every release?
|
Is it a manual effort or are there
any automated tools used?
|
How
often the team monitors the release?
|
% Patch/FIx Releases in Prod
|
Is there any patch releases are
pushed to prod?
|
How
frequently patches are pushed to prod?
|
Application Performance monitoring
|
Is there any monitoring tool that
triggers alerts?
|
Is it
a manual effort or are there any automated tools used?
|
How often the team monitors the
entire application?
|
Self healing systems/Failover
|
Is
there any ayto-recovery system used?
|
Does the application recover from a
sev1/2 incident automatically?
|
Continuous Feedback and Improvement
|
Log Management
|
Are
production logs captured and used for operational analysis?
|
Is there any log management tool
used to monitor the end-2 end flow?
|
Operational Analysis
|
Do we
analyze the application logs to minimize future failures, issues, exceptions
in the application?
|
DevOps Metrics
|
How is the data collected above is
utilized and used?
|
Incident Monitoring and Alerts
|
Do we
have automated reporting of issues and incidents in production?
|
P1 Incident SLA
|
What is the process for sev1 issue
resolution?
|
What
is the SLA for sev 1 incidents in prod?
|
Real-time messaging
|
What is Real-time messaging in your
team?
|
Are
you following any Messaging technique?
|