Building High Performing Teams & Trusted Customer Relationships
Shelby Cunningham, Director of Professional Services
When I look back at my extra-curricular activities as a kid, I realize they were probably foundational for my current role leading app development teams at Effectual. From tap dancing to cheerleading, piano playing to water skiing, I learned coordination, team-building, creativity, and how to adapt quickly in a “fluid” environment. Most of all, they were both challenging and fun, which is how it feels to lead a large development team for a growing entrepreneurial company.
Tech has always felt like a natural fit for me. I started my career managing marketing and overseeing development teams for software startups, which evolved into having my own strategic consulting agency. Those experiences led to management roles in client success, partner relations, business development, and product marketing –positions that required a balance of tech know-how, business skills, and relationship-building with customers, partners, and employees.
Tech has always felt like a natural fit for me.
I joined Five Talent as a program manager several years before the company was acquired by Effectual. As a custom software developer, we always had a high volume of projects with really diverse use cases. This pipeline and our partnership with Amazon Web Services (AWS) made continual learning a strong cultural value for us. To deliver the best solutions for our customers, we had to keep pace with innovation. This is why our developers hold so many high level AWS certifications and why they continue to pursue this expertise as part of Effectual.
Today, I lead Effectual’s professional services app development team. My responsibilities include driving project management and continuous improvement as well as ensuring we are meeting (and exceeding) the expectations of our customers.
The best part of my role here is cultivating long term relationships with customers knowing that our team can build solutions that will have a real impact on the success of their businesses. From the time we start our discovery process to when we launch a product, I want our customers to know we are partners working towards the same goal. That kind of authentic collaboration yields amazing results, and it isn’t hard when your team includes some of the smartest, most talented people in the industry.
The best part of my role here is cultivating long term relationships with customers knowing that our team can build solutions that will have a real impact on the success of their businesses.
Our team is expanding fast as Effectual grows. Because of this, I am focused on building high functioning teams where people are engaged and challenged in their work but also have time to explore new technologies that interest them. Keeping up with innovation is critical to what we do here.
My new interest outside of work is riding motorcycles with my husband. I’ve got a Yamaha now but have my Harley picked out. It isn’t water skiing, but it’s the perfect metaphor for where I am in my career. Moving fast, enjoying the ride, with miles of open road ahead.
Whether you’re considering a data center expansion, full migration, disaster recovery, or building brand new applications, VMware Cloud on AWS can support your modernization efforts while removing the risk of transitioning to the cloud.
In this 30-minute live webinar, we’ll discuss real world applications for VMware Cloud on AWS and offer insights for how to make your next cloud project a success.
Bill Roth, Director of Cloud Economics, VMware
Tom Spalding, Field CTO, Effectual
Lunch is on us! All attendees will receive a $15 GrubHub Gift Card.
In this webinar, you will learn…
Which workloads you can move seamlessly to VMware Cloud on AWS
How to estimate the TCO of moving to the cloud
Why a Cloud TCO analysis can help you avoid costly mistakes
Options and best practices to consider before moving to the cloud
Who should attend?
IT decision-makers responsible for reducing costs
IT practitioners who want to propose cost-effective solutions that get approved
C-Level executives who want to better understand the cloud landscape and key cost-drivers
Date: Thursday, February 25th Time: 12 – 12:30 pm PST Virtual Meeting Details: Zoom (link provided by email prior to the webinar) Cost: Free – Register and attend to earn AWS Credits!
JHC Technology, Inc., provides Cloud Professional Services and Infrastructure to support California government agencies. JHC Technology personnel provide cloud readiness assessments, architecture and design services, engineering and migration, DevOps, Cyber Security, and Project Management resources to support the technology solutions that California agencies are requiring.
JHC Technology Services and Infrastructure, an Effectual Company
November 10, 2021
Amazon Web Services (AWS), DoubleHorn, Google, IBM SmartCloud, Microsoft Azure, Microsoft Cloud Services, NaviSite, Rackspace, Terremark/Verizon
Available Products & Services
Cloud Broker Services, Technical Services
920-05 – Application, Infrastructure, Hosting and Cloud Computing Services
About Texas DIR
JHC Technology, Inc. offers cloud broker services through this contract including: Amazon Web Services, Google, IBM Cloud, Rackspace, Navisite, Microsoft Azure, and Terremark/Verizon. Customers can purchase directly through this DIR contract. Contracts may be used by state and local government, public education, other public entities in Texas, as well as public entities outside the state. Resellers are not available for this contract. DIR has exercised the automatic renewal option for this Contract. This renewal extends the contract through 11/10/2021.
How To Order
For product and pricing information, click here or contact or contact Dan Tudahl at (703) 856-6905.
Generate a purchase order made payable to JHC Technology, Inc and you must reference the DIR Contract Number DIR-TSO-4053 on your purchase order.
E-mail or fax your purchase order and quote form to your designated vendor sales representative.
My introduction to computer science was a 7th grade programming class where I learned BASIC. The class opened up a new world to explore that had me immediately hooked. By the time I started high school I knew I was going to follow a career in technology. Some people have a more circuitous path to their vocation, but I was fortunate to discover mine early on and seize the opportunity to build a really fulfilling career doing what I love.
In my opinion, the most important skill in this field is the ability to solve problems through continual learning. Technology is always changing, and the pace of innovation demands constant attention in order to stay ahead of new tools, services, and solutions. That is why 40% of my job every single day is dedicated to learning-and why it is a such a huge part of our company culture.
Technology is always changing, and the pace of innovation demands constant attention in order to stay ahead of new tools, services, and solutions.
Learning and curiosity are values that also extend to my role as a father to five daughters, age 8 to 14 years. By teaching them computer science, I am trying to give them the confidence to learn about technology and apply it to real world situations and challenges. This is one of the ways I stay involved with their personal lives and hopefully prepare them for the future. It also helps me stay relevant because I have to do my research to understand what their world is going to look like when they become adults.
Teaching my own kids about technology has led to other opportunities to inspire the next generation of innovators. To support students in the Bend community, I teach a 5th grade technology-focused STEM program at my children’s school, host local Hour of Code events, and work with the Oregon Department of Education to integrate computer science into the K-12 curriculum. At work, I mentor recent computer science college grads starting their tech careers as part of the Apprenti internship program.
For myself, continual learning includes going after new AWS certifications, training in other cloud platforms, understanding the pros and cons of multiple stacks, testing new services, and keeping current on industry trends. However, the best opportunities for ongoing learning is working with our really talented developers on a broad portfolio of diverse projects.
Our professional services team typically has 40+ projects underway at any given time (IoT, mobile apps, web apps, big data, system integrations) that use 10-12 languages and multiple cloud providers. Even if I am not working directly on a project, I meet with my technical leadership team every week to review what we did, what worked, what did not, and to figure out what we can do better. This gives me a chance to learn alongside them and gather lessons learned as reference points for when I am talking to customers or recommending a new architecture.
For in-depth analysis, I like to pick a topic such as AI and do as much research as I can to understand what the top 5 vendors offer, the benefits of their solutions, the use cases, and the lessons learned thus far. I also follow a dozen blogs that cover new design patterns so I can compare technology stacks and spend at least an hour a night researching how to stay forward thinking on cloud native architecture.
If you want to deliver a relevant, valuable technology solution, you have to start by understanding the problem you are helping your customer solve.
Still, it is not enough to be a technical expert. As professional services providers, our job is enabling business outcomes with measurable results. If you want to deliver a relevant, valuable technology solution, you have to start by understanding the problem you are helping your customer solve. This includes pain points, opportunities, target audiences, business requirements, the competitive landscape, and more. That is why our solutions architects and developers are skilled technologists as well as big picture thinkers interested in how businesses and market dynamics work. I encourage us to ask WHY we are building something as much as HOW.
After spending the last 15 years focused on professional services, I am excited to embark on the next chapter of my career. Working with Effectual’s Modernization EngineersTM is giving me a whole new understanding of the life cycle of a technology solution. I am gaining a more comprehensive view of how to properly manage and monitor the solutions we build in a cloud environment for the long term. The collaboration is making me a better architect and a better technologist, with more learning ahead.
Learn how to run your Microsoft .NET applications and workloads on AWS for greater reliability, faster performance and improved security, with access to the deep functionality of AWS cloud services. Explore the benefits of using containers to accelerate your modernization journey from monolithic to microservices architectures.
Overview of .NET on AWS
Getting started with AWS tools
Hosting .NET apps
Migrating .NET workloads
Containerizing legacy ASP.NET apps with Amazon ECS
This event includes a hands-on Modernization Lab on containerizing legacy ASP.NET applications using AWS App2container.
SpiraLinks: Rapid migration to AWS unlocks new cloud-native capabilities
SpiraLinks offers tailored consulting services for projects, technical event, and implementation management to Fortune 500 companies, including designing, installing, and hosting secure web-based systems for human resources, compensation, and finance teams. The company’s FocalReview® planning suite is a leader in compensation and performance management, supporting customers in the US and beyond.
Driven primarily by the upcoming consolidation and closure of the data center hosting its product platform, SpiraLinks had made the strategic decision to migrate its infrastructure to the AWS Cloud. This included three application servers, a legacy Oracle database environment, and an older standalone Windows application. The company also had several virtual machines that were being retired by their MSP.
SpiraLinks recognized that a successful migration would provide an opportunity to modernize its technology stack and leverage new AWS capabilities to better serve its customers. However, without the internal resources to accomplish the move, the company needed to engage a partner with the technical resources and expertise to achieve the migration.
Benefits of the AWS Cloud
The company chose to migrate to an AWS environment to increase efficiencies, improve security and compliance, and optimize costs. In addition, the SpiraLinks wanted to access new AWS Native services to modernize and evolve its business.
Outsourcing Migration Expertise to Effectual
To achieve these business objectives, SpiraLinks partnered with Effectual to lead its migration and modernization efforts. Effectual is a cloud first, security first managed and professional services company and AWS Premier Consulting Partner with deep expertise leading complex migrations and managing modern cloud environments across VMware, VMware Cloud on AWS, and native AWS environments.
Solutions & Outcomes
Completed a full migration of customer-facing applications from on-prem infrastructure to a new, modern, secure AWS environment in less than a month.
Deployed all new modern Linux and Windows servers in separate VPCs for improved security
Configured Amazon Elastic Block Store (EBS)for the three Linux EC2 instances hosting Wildfly (formerly JBoss)
For Oracle database server:
Migrated all data from legacy Oracle environment
Upgraded and deployed database into a new Amazon Relational Database Service (RDS), allowing for adoption of Session Manager for accessing application servers (improving security and decreasing costs) and providing added functionality with real-time performance insights
Increased the security layout of the data environment by isolating it in its own private subnet and restricting access
Restricted access via approved ports from application servers
Deployed to a single AWS RDS instance with individual database schemas
Replicated and enhanced mail sending capabilities to utilize Fluent Ltd. mail relay service.
Increased security due to inherited ISO certification from AWS.
Created an AWS Identity and Access Management (IAM)group and defined the IAM policy to provide SpiraLinks developers with access to the AWS Systems Manager Agent(SSM). Once the IAM groups and policies were configured, shared initial login credentials with the primary SpiraLinks contact and configured Multi-Factor Authentication (MFA) to enhance solution security.
Deployed and configured CloudTrail and CloudWatch EC2 log streams to monitor instances, and also configured email alerting for these services.
Configured Amazon Data Lifecyle Manager (Lifecycle) to take snapshots, with a rolling 14-day retention period.
Established a clearer understanding of data needs as well as the specific benefits of AWS environment and services in order to make informed choices.
Through its partnership with Effectual, SpiraLinks was able to achieve a rapid migration of its infrastructure to the AWS Cloud and avoid unexpected downtimes associated with the closure of its MSP’s data center. The migration to an AWS environment provided opportunities to improve security, increase efficiencies, and optimize costs while opening new pathways to modernizing using AWS native services and capabilities.
Moving forward, SpiraLinks will utilize the newer, more secure AWS environment for its many tools and benefits in accordance with the constantly changing business and operational requirements of the SpiraLinks client base. Specifically, compliance and data protection/privacy will be evolving challenges for SpiraLinks and the client base. The AWS environment has been chosen as an excellent “base of operations” to meet those challenges.
SpiraLinks will continue to work with Effectual as a Modernization Service Provider to utilize their expertise in addressing the company’s long-term goals and challenges. In addition, SpiraLinks and Effectual have developed an evolving roadmap that includes further modernization efforts to increase automation, availability, reliability, and security – further establishing the position of SpiraLinks as an industry leader.
About Time Tours: Guiding a Successful SaaS Journey
About Time Tours is a Pacific Northwest startup redefining how the real estate industry plans, organizes, and coordinates home tours between agents and homebuyers. With market expertise but only a general business idea, the company asked us for help developing a SaaS-based solution. We guided them on their SaaS journey from a basic app concept to a scalable production-ready launch using AWS SaaS services and best practices.
As a startup, About Time had already identified key pain points facing both realtors and home buyers for scheduling home tours. For all involved, the existing process was time consuming, cumbersome, and fraught with unnecessary complexity. About Time saw an opportunity to streamline scheduling and communication and capture feedback. The company also wanted to maximize the market opportunity and go to market as quickly as possible.
Building the business case & defining the product vision
Given almost 50% of our professional service engagements are SaaS focused, we have deep experience implementing the SaaS business model for clients. We started with About Time by embarking on a full discovery process, beginning with building a well-defined business case for their solution and outlining their product strategy.
This included evaluating customer pain points, developing user stories, and creating a seamless UX/UI experience. We also conducted a competitive analysis of off-the-shelf solutions to determine what problems they solved, how they solved them, and their challenges. After establishing the business case and product vision, we built a series of wire frames showing functionality of features and workflow before moving on to mockups of the app.
Aligning the MVS with AWS SaaS best practices
For SaaS clients, defining a Minimum Viable Service (MVS) always poses the greatest challenge. It is also the most critical stage on the SaaS journey where resources are concerned, as you can easily over architect your solution and run up costs. We worked with About Time to decide on the right MVS, knowing they would receive important feedback after going live that would likely change the app in future sprints.
Once we had defined the MVS, the AWS SaaS Enablement Framework provided a clear, thorough process for us to evaluate tenancy, security compliances, and compare cost models against the company’s revenue objectives. We also helped About Time prepare documentation and collateral in support of their efforts to secure investor funding.
From development to launch -Leveraging the Well-Architected Framework
In the next phase, our development efforts followed an agile process with milestones sprint by sprint and continual, transparent communication with About Time’s founders and investors. We used the Well-Architected Framework to ensure we were properly evaluating tradeoffs and applying cost optimization strategies when it came to reliability and security. We also segregated their personally identifiable information (PII) data in a multi-tenant environment to meet security compliances.
In addition, we built their app 100% on serverless so it can scale rapidly as their user traffic increases and utilized a pay as you go model to keep costs per user in line with their profit margin and revenue expectations for sustainability and growth.
For testing, our team conducted performance tests to ensure the app could handle expected traffic and security tests to confirm there were no hacking activities. We also held an informal gameday so that there is support documentation in place in case the app goes down during a live environment. Last, our SLA with the company sets expectations regarding our response time and steps we will take to ensure they are up and running quickly.
At launch, About Time’s final mobile and web app represents a highly scalable SaaS solution capable of growing with market demand without compromising on security and cost. Our next steps include capturing feedback and optimizing features and workflows to keep customers happy and satisfied with their solution.
Working with the Effectual team to refine our MVS gave us an objective view of how to align our revenue goals with the right cost model so we could take an informed approach to choosing the best SaaS strategy. As we scale, we know we can meet our business objectives and deliver a high-quality customer experience.
Empowering Marketers and Driving Customer Engagement with Amazon Pinpoint
In an increasingly virtual world of remote work, online learning, and digital interfacing, successful customer engagement can differentiate you from competitors and provide deeply valuable insights into the features and innovations important to your users. A well-designed, well-managed user experience not only helps you gain market share, but also uncovers new revenue opportunities to grow your business.
At Effectual, we begin projects with an in-depth discovery process that includes persona development, customer journey mapping, user stories, and UX research to design solutions with engaging, meaningful user experiences. Post-launch, your ability to capture, iterate, and respond to user feedback is just as essential to your success.
In our experience, many SaaS-based companies simply miss this opportunity to stay engaged with their customers. Reasons for this include the complexity and cost of designing, deploying, and managing customized marketing campaigns across multiple channels as well as the lack of real time data analytics to inform them. The result is a tidal wave of generic emails, poorly-timed push notifications, and failed initiatives that impact customer retention and engagement.
Amazon Pinpoint is a scalable outbound and inbound marketing communications service that addresses these challenges and empowers marketers to engage with customers throughout their lifecycle. The service provides data insights and a marketing dashboard inside the Amazon Web Services (AWS) admin console for creating and managing customized communications, leveraging automation, data analytics, filters, and integrations with other AWS products and third-party solutions.
Easy to use and scale
Manage campaigns from a user friendly marketing dashboard
Scale reliably in a secure AWS environment
Targeted customer groups
Segment audiences from mobile and web application data or existing customer list
Customized messaging across email, SMS, push notifications
Personalize content to engage customers using static and dynamic attributes
Create customer journeys that automate multi-step campaigns, bringing in endpoints from your app, API or directly from a CSV
Engage customers with targeted emails and push notifications from the AWS admin portal using rich text editor and customizable templates
Set up customer endpoints by user email, phone #, or UserID # to track user behavior within your app
Use real time web analytics and live data streams to capture immediate feedback
Measure campaign data and delivery results against business goals
Integrations with other AWS services and third-party solutions
Integrate Amazon Pinpoint with intelligent services such as Amazon Connect, Amazon Personalize, and Amazon Forecast
Use with existing marketing solutions such as Salesforce and other third-party application
For marketers, Amazon Pinpoint is a powerful tool for improving digital engagement – particularly when integrated with other AWS services that utilize machine learning and live stream analytics. Organizations that invest in designing engaging user experiences for their solutions will only benefit from continually improving and innovating them.
Have an idea of project to discuss? Contact us to learn more about using Amazon Pinpoint to improve your customer engagement.
Tricon American Homes: IT modernization helps industry leader keep pace with accelerated growth
Tricon American Homes (TAH) is the fourth largest single-family rental company in the US. Founded in 2012, the publicly owned company has a portfolio of over 20,000 homes in ten states. As one of the country’s fastest-growing real estate companies, Tricon has gained a competitive advantage by offering highly responsive, personalized customer service and translating it into profitable long-term relationships.
Since 2016, the company has more than doubled its rental home portfolio. This accelerated growth highlighted the need to formalize and streamline processes, reduce costs, and optimize its operational efficiencies. With properties spread out over large geographic areas, Tricon was also in search of scalable solutions for managing, servicing, and maintaining their homes as well as a responsive communication platform for delivering a high-touch, seamless experience to its residents.
Challenges to pursuing these strategies included limited internal development resources as well as a lack of off-the-shelf solutions for the single-family vertical market. As the company expanded, Tricon partnered with a specialized team of solutions architects at Effectual to integrate their business requirements with DevOps expertise and take advantage of evolving Amazon Web Services (AWS) solutions.
Improving operational efficiencies while scaling teams and services quickly
Combining multiple data sources to create complete and holistic reporting
Innovating continuously to optimize costs and meet market demand
Creating a DevOps culture focused on automation, cross department communication and collaboration
During the last four years, Effectual has supported these goals by designing, developing, and deploying numerous solutions for the company leveraging the AWS Cloud. These include applying IoT capabilities, integrating smart home technologies, and utilizing AI/ML managed services for revenue enhancements.
Solutions & Benefits
Operational Efficiencies One of Effectual’s first projects was to streamline Tricon’s existing rental process and leverage automation to integrate existing administrative functions with custom business applications. In addition, the team developed and launched a 200-home smart home pilot with BeHome247, a cost-saving program Tricon is rolling out to its entire portfolio.
Continuous Integration – Continuous Deployment (CI/CD) To speed time to market deployments, increase reliable releases, and provide a secure environment, the team also created a CI/CD stack and pipeline that aligned new feature requests from ideation to deployment.
Performance & Functionality In order to reduce AWS costs and increase scalability, Effectual streamlined Tricon’s application payments, built a performant dashboard, and deployed AWS S3 for highly scalable cloud object storage.
Monitoring & Logging Last, by integrating all aspects of custom applications with CloudWatch, Tricon can now easily monitor and quickly troubleshoot issues without affecting their customer experience.
The success of these initial improvements has led Tricon to further expand its partnership with Effectual, including exploring new AWS services and developing additional custom applications to better serve its residents. The company is rapidly become an industry leader in new technologies.
“Effectual has been an extension of our team for several years, and we appreciate their focus on implementing scalable, innovative, and well-architected solutions in partnership with us. They continually go above and beyond to ensure that new projects are successful by asking questions and clarifying assumptions to truly understand our business objectives. We utilize their knowledge to evaluate new technologies and services to ensure that our technology stack is optimized.”
– Dawn Dalton, VP of Business Systems
Implementing Modern Cloud Management & Optimization
As the team addressed Tricon’s development requirements, it recognized the company was going to need ongoing management to monitor and maintain the security and performance of its AWS platform as well as to identify opportunities for cost optimization and business intelligence strategies.
With a growing estate of applications running on AWS continued, it became apparent that opportunities existed to improve the maintenance and security of Tricon’s environments. As a cloud-first, security-first Modernization Service Provider, Effectual provided the experience and expertise to keep the company on a path of continued innovation.
In particular, Tricon had experienced disruptions within critical business systems and wanted to improve their response time with greater visibility into what was causing errors. By establishing automated monitoring alert systems, Effectual has helped Tricon to respond quickly and resolve issues as they occurred, reducing downtime and improving customer experience.
“We’re coming to resolution much faster now on issues. Before working with Effectual, it was taking us longer to figure out the root cause.”
– Gregg Knutson, Sr. VP of Information Technology
With consistent reporting, Effectual’s delivery team has also been able to uncover patterns affecting Tricon’s costs. During a recent quarterly business review, Effectual reviewed the company’s 6 month cost trends and identified an unutilized RI (Reserve Instance) that had become orphaned. Resolving the issue helped Tricon take advantage of the significant discount RIs offer – a key strategy for cost optimization. This proactivity is one of the most important benefits of having a long-term partnership with a Modernization Service Provider.
As a trusted advisor, Effectual’s overall goal is to set Tricon on a path of scalability and growth with the confidence it can securely and reliably serve its customers. This includes aligning the company’s business goals with new tools, methodologies, and strategies to support their growing business.
“We try to be a forward-thinking organization in terms of technology, and really want to leverage modern IT systems. With this partnership, we’ve deepened our ability to meet the growing needs of our technology roadmap.”
Building secure, scalable SaaS solutions with the AWS Cloud
Leverage the power of AWS innovation to develop successful SaaS applications that scale to meet your customer and market demands – without compromising on security and cost. Well-Architected SaaS solutions can increase your agility, reduce time-to-market, and expand revenue opportunities. In this workshop, we will cover:
Business best practices
User onboarding and customer engagement
Revenue models & cost objectives
Monitoring & KPIs
Pros/Cons of different tenant isolation patterns
There will also be two 15 minute breakout sessions to applythese concepts to a fictitious case study.
REGISTERand attend to receive a $25 AWS hosting credit and $15 GrubHub voucher
Zolo Media: Integrating custom solutions to activate and capture a media-hungry regional market
Zolo is a media broadcaster and production company based in Central Oregon. The company provides broadcast and advertising solutions through local, network, and original programming to Central Oregon viewers. Zolo is owned by Telephone and Data Systems, Inc. [NYSE: TDS].
Recognizing that viewers are increasingly consuming content online, Zolo positioned itself to offer viewers a robust online viewing experience with daily video content, live streaming, and real-time weather information. The company asked Effectual to help them evaluate approaches for building their new online media platform.
Strategic consulting reveals clear requirements and high level objectives
Effectual and Zolo had a collaborative, in-depth discovery process that uncovered the complex business requirements and technical needs for the new platform. Zolo was particularly concerned about meeting the rigorous corporate security, legal and risk compliance requirements of its parent company TDS.
Scope, budget and project phasing recommendations matched Zolo’s requirements and business objectives.
Complied with all security, legal and risk requirements of publicly-traded parent company.
“Effectual has been an amazing partner over the past year – always finding a way to make “what if” happen and showing us the possibilities.”
– Michele O’Hara / Marketing & Creative Services, Zolo Media
Amazon Web Services (AWS) empower reliable performance and website scalability
As a certified AWS solutions architect, Effectual saw quickly that AWS could provide Zolo performance reliability for its streaming content while allowing the company to dramatically scale its offerings.
Caching and scalability of up to 2 billion unique visitors per month
Consistent, uninterrupted streaming content
Hosting bandwidth and cost scales with site traffic
Ability for multi-authors to add content
Custom integrated technical solutions build user community
The team built a custom website design responsive to desktop, tablet and mobile that complemented the platform offerings
Xenon, Inc: IoT Proof of Concept Accelerates New Market Opportunities
Xenon, Inc. is a custom hardware provider offering full-service engineering, integration, and field service solutions for the oil and energy industries. The company provides process and environmental analytics, industrial instrumentation and automation, and electrical systems.
Though Xenon primarily serves industrial markets, the firm was approached in 2018 by a new customer interested in applying their industrial engineering background to building IoT solutions for optimizing home maintenance, monitoring, and asset protection. An institutional single-family residence company with a portfolio of thousands of homes, the client was particularly interested in testing automated door locks, water sensors, and other smart devices for secure access and efficient maintenance. Their proposed plan included deploying devices in vacant properties each month with a three-year installation phase.
Partnering with Xenon provided an opportunity to explore and validate the impact of installing IoT smart home solutions for improved customer experience and reduced operational costs. For Xenon, the project presented a new market outside of its industrial focus. To respond, they needed a proof of concept to test in the first 200 homes and present to executives.
Leveraging expert advice for faster proof of concept
Xenon began building the IoT platform in Amazon Web Services (AWS) on its own, but soon encountered issues. As hardware engineers, they realized they were outside their core competency and needed help from experienced solutions architects on software integration with their client’s property management system. They engaged Effectual to review their existing architecture and implement Well-Architected best practices.
Xenon’s primary challenge was creating a cost-efficient cloud architecture that could scale. When the Effectual team conducted an initial review of the company’s environment, we confirmed the existing software layer would require fundamental changes to meet their cost requirements. In addition, our evaluation revealed the platform was built on one computer with no staging environment and no redundancy. This existing environment jeopardized the long-term reliability and scalability of the platform.
Based on this analysis, our team estimated Xenon would quickly out-capacity their existing environment at 100 homes. This was insufficient, as they needed to prove they could scale rapidly to service the company’s expanding property portfolio. Effectual also felt Xenon’s small development team could benefit from mentoring and guidance on key concepts and AWS IoT Core best practices.
Key recommendations and outcomes included:
Built a scalable, reliable proof of concept that met the client’s business requirements and budget
Confirmed AWS as the right solution for expanding their offering
Established DevOps best practices and trained internal team on processes
Educated company on costs and complexity of creating an IoT solution on a traditional infrastructure with EC2, load balancers. Showed them the significant benefits of using a serverless framework to process IoT events from Amazon Kinesis and device command management.
Developed 187 AWS Lambda functions for an estimated 40,000,000 events per month.
Implemented Amazon Kinesis to collect, process, and analyze 60,000 incoming records per day (30 MB of streaming data per day) to provide reliable, real-time insights and rapid response capabilities.
Deployed AWS API solution with an advanced logging and control layer for Xenon’s large scale IoT system to handle a high volume of burstable requests. Designed one gateway to ingest IoT alarms and events, and another to receive commands from external systems and applications.
Implemented Amazon DynamoDB as the primary storage mechanism for scalability with all tables using On-Demand for capacity control.
With Effectual’s help, Xenon responded quickly with a functional, reliable proof of concept that addressed their client’s pain points and met their business requirements. They validated AWS as the best cloud solution for propelling their project forward and gained a solid understanding of AWS IoT services.
Results & Next Steps
For their client, the project provided a better grasp of the costs and resources needed to deploy smart home systems in their properties. It also revealed what checks and balances they need to put in place for their operations.
From Effectual’s perspective, these outcomes are precisely what a successful proof of concept project should accomplish. If the client does decide to roll out these systems to its entire portfolio, we look forward to helping Xenon revisit its current configuration with some new approaches to further unlock the potential of the AWS Cloud.
Wingo IoT: AWS IoT Solutions Position Startup for Rapid, Secure Scalability
Wingo IoT is an Oregon-based startup that integrates inexpensive IoT and legacy automation systems into one intelligent solution for industrial applications. Its key value proposition lies in collecting critical data for operational analytics, AI and ML modeling, and insurance premium and claim reductions. Established in April 2018 by an experienced technical team, Wingo focused its early development efforts on local sensor networks and isolated edge devices for data collection. The company’s hybrid IoT solution included 100% offline monitoring at sites and low-cost methods for collecting and managing facility data.
From the beginning, Wingo was aware their initial cloud architecture would require major improvements to meet stringent availability and security requirements for modern enterprise applications. A growing pipeline of large industrial customers motivated CTO Glynn Fouche to approach Effectual for a Well-Architected Framework Review as well as expert advice on Amazon Web Services (AWS) IoT solutions.
Starting with Well-Architected best practices to build long term success
Fouche recognized Wingo needed to properly leverage cloud services in order to best serve their customers. In particular, he wanted to set the young venture up for success from the start by aligning Wingo’s development process with the 5 Pillars of the AWS Well-Architected Framework.
As an AWS Advanced Consulting Partner and authorized Well-Architected reviewer, Effectual frequently helps early-stage companies leverage cloud-based solutions for projects ranging from proof of concept to full-scale custom software development. In this case, it was clear that with few developers, limited resources, and impending customer rollouts, Wingo was on a tight schedule to identify critical issues for remediation, improve real-time reporting, and operationalize its development process.
Given the company’s aggressive timeline, our team of solutions architects completed a thorough Well-Architected review and remediation in less than three months. During the process, we uncovered 34 high-risk issues requiring attention.
In the process of addressing these key issues, Effectual’s contributions include:
Developing cost predictions for company revenue model
Adopting a flexible consumption model to reduce development to cost ratio and increase product margins
Designing and implementing DevOps process for long term scalability
Establishing data storage plan leveraging a combination of Amazon DocumentDB, Amazon S3, AWS Glue and Amazon Redshift for quickly indexing data with instant access
Ensuring security compliance in a multi-tenant environment by isolating sensitive data
Creating NOC dashboard using AWS Lambda for real-time monitoring and business logic for pulling analytics
Deploying Amazon CloudFront to move small json payloads of dynamic content
Leveraging API Gateway as the medium for mobile and web apps to trigger backend API services in AWS Lambda
Providing security and disaster recovery analysis as well as recommendations for a secure, highly available, and fault-tolerant architecture
In addition, implementing Well-Architected best practices has strengthened Wingo’s confidence it can serve larger customers and meet their strict business and compliance requirements. Based on past experience, Fouche believes Wingo is much better prepared to handle comprehensive due diligence and security audits. The review process and documentation will also have a significant impact on the company’s ability to raise capital and could add significant value in the event of a purchase.
In collaboration with Effectual, Wingo’s next steps include documenting security practices as well as failover and recovery recommendations for performance reliability. These steps are critical as the company develops its cloud-based data architecture, user interfaces, and API gateways for external integrations.
Results & Next Steps
With the Well-Architected review complete, Wingo is now positioned to approach both new customers and potential investors with greater confidence in its ability to receive, process and store data in the cloud and offer powerful data insights for driving optimal business outcomes.
Warm Welcome: Replicating the SaaS delivery model with a smart Proof of Concept (POC)
With a history of successful SaaS ventures for the photography and real estate industries, entrepreneur David Jay launched Warm Welcome as a Proof of Concept (POC) in early 2019. The product delivers highly personalized video messages through email to support customer onboarding and retention. After investing nine months to gather user feedback, Jay had developed a clear pricing model, a list of MVP features, and go-to-market strategy. However, he needed Effectual’s help refactoring the POC to address security and reliability concerns to make the solution production-ready.
Evaluating trade-offs, defining priorities
Highly skilled at building strong, loyal user communities, Jay and his team are adept at responding to user requests, defining focused MVPs, and gathering valuable customer feedback. With Effectual’s support, they have also learned how to use the Well-Architected Framework to evaluate trade-offs and determine priorities for their POCs.
For Warm Welcome, the team decided that time-to-market in the POC phase was a priority. Their goal was to quickly capture user feedback in order to understand the product’s business value.
Aligning pricing and marketing strategies
The first version of Warm Welcome was a small, low fidelity MVP tightly focused on solving the customer’s biggest problems, which were closing a sale and onboarding a new client. Based on analytics, user surveys, phone calls, and focus groups, the company gained key insights into the value of the product. This helped them align their messaging and marketing with the needs of their customers.
In addition, they carefully tracked their actual costs, allowing them to build a pricing model to accurately fit their cost model.
Refactoring architecture for reliability and security
While reliability and security were acceptable trade offs during the POC phase, they needed to be addressed prior to moving into production. Effectual began refactoring the POC by conducting a Well-Architected Framework Review (WAFR), resulting in a re-evaluation of the initial tradeoff decisions. In addition, Effectual offered the following recommendations:
Secure the environment to autoscale using EC2, ElasticBeanstalk, AutoScalingGroups
Set up a CI/CD pipeline with parallel environments to increase agility and lower risks
Leverage additional AWS tools and services including Cloudfront, S3, Aurora RDS, and ElasticTranscoder
Warm Welcome is one of many projects Effectual has worked on with Jay over the last several years. Throughout the client relationship, Effectual has provided strategic advice and technical expertise through all phases of discovery, development, and deployment.
Verdant Web Technologies: Seamless Integration with Amazon EC2 for Microsoft Windows
Verdant’s management software solutions track, access, and update facility Environmental Health & Safety (EH&S) compliance and sampling information. With a growing customer base, the company was beginning to confront scalability, reliability, and performance challenges that could not be solved with their existing on premise infrastructure. Verdant President Ron Petti engaged Effectual to conduct a Well-Architected Framework review and support their migration to the AWS cloud.
As an enterprise level company, Verdant wanted to migrate to AWS with their existing Microsoft technology stack. This was a requirement for two reasons: 1) All of their technical engineers already used Microsoft (.NET and SQL Server); and 2) Verdant’s industry clients had tools integrated with Microsoft and were loyal to their solutions. With several Microsoft certified developers, we were confident of a seamless integration with AWS services.
We rewrote Verdant’s entire on-premise workload to AWS using Amazon EC2 for Microsoft Windows Server, building a scalable framework that addressed their business challenges. To do this, we leveraged IIS and the .NET framework. We kept the company’s SQL Server but hosted it on Amazon RDS for SQL Server to improve capacity, decrease costs, and reduce database administration. We also automated their deployment using Jenkins and Elastic Beanstalk. Like their enterprise clients, they were using Active Directory for user authentication which we were able to easily integrate with SAML.
In the end, Verdant was able to stay with a Microsoft technology stack their internal team and customers trusted and were familiar with while taking advantage of the compute capacity of the AWS cloud. Because we began with an existing framework, we were also able to complete the migration far more quickly and focus on adding value with our customized solution.
Verdant Web Technologies: AWS tools improve scalability, profitability, and customer experience
Verdant offers management software solutions to track, access, and update facility Environmental Health & Safety (EH&S) compliance and sampling information.
With a growing customer base and a maturing product, Verdant was starting to encounter big DevOps and infrastructure challenges that threatened to slow its market momentum. The company’s migration of thought and concept demanded a far more scalable model. To the Effectual team, it was clear that the AWS platform could help them pivot and evolve.
Standardized architecture improves DevOps
Verdant’s primary pain point was architecture. With six different code bases unique to each client, the company updated code changes manually, published them out to 10+ web servers, and ran its SQL scripts on multiple databases. The process was overwhelming their team, impacting scalability, and preventing them from writing new features. The company’s IP also lived with a single developer, creating some vulnerability. Our team immediately got to work rewriting the company’s software with multi-tenant support, allowing different organizations to manage their data separately but with a standardized code base.
Streamlined DevOps by automating the deployment/development process with a build server and rapid deployment tools
Created a faster, more reliable migration to the AWS Cloud
Leveraged AWS for greater security and global redundancies to safeguard against potential downtimes
IP knowledge is now shared broadly by Verdant’s entire team so the company is no longer reliant on one person to protect its IP
The ability to scale rapidly to meet customer demand
“Effectual has been an amazing partner in the development of our enterprise platform which is now our life blood. Along with their responsiveness, solution engineering depth and capabilities we appreciate their tight management of project budgets and schedules. Effectual is a valued resource and critical part of the Verdant Team!”
– Ron Petti / President, Verdant Web Technologies
Eliminating hardware lowers cost of customer acquisition
Before deploying AWS, it took Verdant weeks to onboard new clients with a process that required significant hardware investments. Infrastructure was a fixed asset regardless of the number of clients. Our solutions turned infrastructure into an operating cost and eliminated hardware altogether.
Reduced new client onboarding from 2 weeks to 1 hour
Eliminated need for costly hardware
Decreased customer acquisition costs
Scalable solution allows for fast response to market demand
For Verdant, the timing for the project couldn’t have been better. Shortly after its completion, the company’s client base exploded overnight when schools around the US were compelled to perform extensive drinking water testing in reaction to the national crises in Flint Michigan. The revelation resulted in stricter reporting requirements and EH&S monitoring across Oregon, driving sudden intense demand for Verdant’s software solutions. With Effectual’s help, the company was well positioned to capitalize on incoming project opportunities, which resulted in a national award (with Environmental Business Journal)
AWS solutions such as Elastic Beanstalk support continuous development and innovation and help Effectual manage multiple application environments for the development/testing/release cycle
Increased customer satisfaction with ability to quickly add new functionalities
Tourvast: Building SaaS Solutions Using Scalable Innovation
Tourvast is a Software as a Service (SaaS) provider with a marketing platform that offers real estate photographers tools for creating property presentations and virtual tours that showcase their skills, leverage their art, and build their business. The platform also offers agents the opportunity to enhance their brand across social networks with high end, quality photography and video assets.
While the platform had been in existence for over a decade as licensed software, Tourvast executives wanted to evaluate the company’s intellectual property and consider options for writing their own application for greater usability. With new business requirements and a new go-to-market strategy, they contacted Effectual for help with their decision-making process and next steps. Our team provided insights and strategic advice and ultimately implemented a more scalable platform based on the secure, reliable infrastructure of the Well-Architected Framework.
Recalibrating the pricing model
In 2016, Effectual solutions architects began a discovery process that included wire-framing and architecture planning. This exposed one of Tourvast’s primary challenges, which was the inability to scale its pricing model. Due to the unpredictability of its customers’ large media files, the current architecture was not consistently covering costs.
Conducted an in-depth revenue modeling analysis to identify average costs based on number of photos uploaded as well as the number of videos, pdfs, and other assets created.
Designed new architecture for cost objectives with pay-for-use pricing to reduce capital expenses.
Improving performance, scaling for demand:
In addition, tenant activity was slowing performance and impacting overall customer satisfaction and retention.
The existing workflow began with a transaction outside of the platform between a real estate agent and photographer to secure photos for the creation of marketing deliverables. Photographers paid a subscription fee for a specific number of properties in advance and banked them like a credit system. After taking photos of the agent’s identified property, they uploaded their images on the website in order to organize them into deliverables such as slideshows, virtual tours, flyers, and more. Once complete, they provided their realtor customer links to those assets. Upon the agent’s approval of the copyrighted materials, they pay the photographer the invoiced amount through Tourvast to release the media for use.
The challenge was that each time a photographer uploaded their image files, the software would immediately resize them and create a slideshow. This process would take up the site up to 10 minutes while the photographer waited for it to complete. At the same time, it froze platform functionality for all customers on the site.
Leveraged serverless architecture using S3 and AWS Lambda for media and multi-tenant loads, resulting in greater flexibility and stability.
Implemented CloudFront for streaming videos to deliver content to end users with lower latency.
Deployed a blue-green architecture on AWS creating a continuous integration/continuous deployment (CI/CD) pipeline, including up to 10 servers for burstable traffic.
Code is now developed and deployed to an AWS Elastic Beanstalk environment, with two separate, but identical, environments (blue and green) to increase availability and reduce risk. This allows the application to continues to run seamlessly while new code is deployed without impacting the user experience.
Implemented DevOps strategies and best practices with parallel development, testing, staging, and production environments.
Ensured that no development takes place in production
Created a testing environment for internal QA
Enhanced reliability with a staging environment built for “friends and family” releases with a copy of production data scrubbed for security reasons (with scale of data to mimic what happens in production)
Today, Tourvast is a SaaS company that owns its own intellectual property, with full control over its roadmap. With support from Effectual, it owns its maintenance backlog and understands its third party dependencies and costs. Last, our team continues to help the company innovate and build improvements using proof of concepts fueled by cost-effective AWS tools.
Robert Axle Project: eCommerce website redesign increases product sales and customer connections
Robert Axle Project is the authority on 12mm and 15mm thru axles for bikes. They provide the highest quality products that allow families, adventurers, commuters and recreationists to enjoy traveling by bicycle. Sales are international, online through its e-commerce website as well as through dealers, distributors and OEM partners.
As the “RAP” products have grown in popularity, and in “fit” complexity with an expanding variety of different bike styles and brands, company founders recognized they needed a stronger e-commerce website platform.
RAP has a strong customer following and sense of their community through social media channels, trade shows and industry connections. During website planning we identified 4 different customer personas to message to about not only about the product, but also in the different narratives and imagery that could be shared around actually ‘having’ and using the product from RAP.
Thoughtful website copy and messaging was matched with imagery to draw visitors into a compelling company narrative about experiences, recreating and possibility with the RAP product.
The transition in content from heavy product and features focus to more customer benefits and aspirations orientation with the new website increased website engagement respectably – with 110+ % increase in page views and 23% decrease in the site-wide bounce rate.
“Effectual worked hard to understand our business needs and develop custom features that increased traffic, conversions and engagement with our customers. With excellent project management and communication throughout the process!”
– Katy Bryce & Chris Kratsch / Founders, Robert Axle Project
Custom fit selector streamlines visitor experience
An increasing complexity in matching product ‘fit’ to increasing variations of bike and trailer types, RAP sales support knew there had to be a better way. They asked Effectual to build a custom “fit selection” tool compatible with the new website CMS.
Automating the ‘fit’ process has facilitated customer ease with securing the right product through the website, dramatically reducing sales support calls and product returns.
New website at the hub of a fuller engagement platform increases visits
Actively engaged with their community, RAP’s new website supports this conversation – through regular story content (blog posts) and integration with social media feeds. Fresh, regular and fun content that’s attractive to search engines and industry forums brings a 150% increase in website traffic.
Online product sales have increased 125% since the website’s launch.
Pole Pedal Paddle: Cost Optimization Strategies Maximize Impact for Local Non-Profit
Held in Bend, Oregon, the SELCO Pole Pedal Paddle (PPP) event is a popular annual multi-sport relay race benefiting Mt. Bachelor Sports Education Foundation (MSBEF). Over 2500 amateur and pro athletes compete solo or as a team in the race each year. The six legs of the race include Alpine skiing, Nordic skiing, biking, running, a canoe/kayak/SUP leg, and a sprint to the finish. Race Manager Molly Cogswell-Kelley asked Effectual for a custom solution to improve race registration, team management, and results reporting as well as to reduce PPP costs.
One of the main cost drivers was an expensive yearly subscription service with a third party provider. With two races (the adult PPP and the kid’s PPP), the event was paying for two subscriptions. In addition, even though their production only ran 4 ½ months each year, they were being charged for an entire year. The off-the-shelf service also lacked needed functionality for managing PPP’s different categories and unique team structures. In addition to growing costs, the organization had a new legal requirement for attaining parent signatures for child waivers.
Applying Well-Architected Framework best practices for cost optimization, our team worked on several strategies targeting the PPP production environment and development process to address business requirements and meet cost objectives.
First, it was clear that an annual subscription was a poor pricing model for the once a year event. We shifted to a more flexible, pay-as-you-go solution with AWS to ensure usage and costs match PPP’s short production timeline. Using AWS allowed PPP to run servers and pay for usage only when their environment was turned on. Given that our development team has nine months each year to develop new features and push out code, we also decided to use spot instances for the development and CI/CD environment, representing a 90% discount.
For the same amount PPP was paying each year to manage its waivers, we were able to build a custom application with far greater functionality and flexibility to accommodate ongoing development needs. Due to the nonprofit’s budget constraints, we focused on the adult PPP race as the first phase of the project. We were able to build on this framework two years later for the kids PPP.
To optimize costs, Effectual leveraged their existing knowledge with .Net membership provider, .Net user provider and the .Net MVC frameworks and libraries. This allowed Effectual to focus only on the custom business logic and leverage out of the box solutions for logins, account creation, registrations, and password creation. This also meant we did not have to re-engineer a security feature, which kept development costs lower. For enterprise reporting and integration, Effectual chose Microsoft SQL Server and IIS.
By consolidating expenses, utilizing spot instances, and integrating AWS services with existing Microsoft technologies, we deployed custom applications that have significantly reduced PPP’s costs and improved their overall user experience year after year.
Several years ago, Effectual worked with BBQ manufacturer MAK Grills on a product ideation project for a new web app giving owners remote operational control of their grills. Prior to our engagement, the company’s outsourced development process had stalled and they needed help salvaging the project. After reviewing their existing code, we were able to address their wishlist and launch the app on an aggressive timeline.
While the app ran successfully for the first few years, the company began to experience performance issues as its customer base expanded. MAK Grills President Bob Tucker re-engaged Effectual for a Well-Architected Framework Review to evaluate their solution, which was crashing daily and shutting down all of their grills.
For the review, Tucker had the following objectives:
Stabilize the production environment
Build an affordable solution using their existing Microsoft technology stack (.NET, SQL Server, IIS)
In our Well-Architected Review, we discovered that the company had recently hired someone to rewrite their firmware. During the rollout, the firmware had 10,000 BBQs sending messages to the MAK Grills website every 5 seconds. This increase in traffic was causing their server to crash at least once a day. They had tried to fix the issue but it was still unresolved. With owners who expected their mobile service to be available 24/7, it was clear the company had a serious customer service problem on their hands.
Actions & Recommendations from the Well-Architected Review
Analyzed the MAK Grills Microsoft server (.NET technology stack with a SQL Server on the backend) to identify what was crashing.
Consulted with their firmware developer to provide guidance on IoT best practices.
Coordinated all of the company’s outsourced engineering teams to ensure they are on the same page in terms of cost objectives and best practices for scalability and reliability.
Our team also recommended that MAK Grills capture their market metrics to understand the business value of their offering. We installed Google Analytics to evaluate customer behavior and created a company dashboard for greater visibility into user data. In addition, we suggested they evaluate switching their business model to monthly subscription pricing (versus charging a $300 upfront cost for the app at time of purchase).
Based on user feedback and customer data, the MAK Grills sales team is now testing a monthly subscription pricing model with new customers. Effectual’s remediation has stabilized a production environment that can scale automatically, and the company can focus on new product innovation to keep its customers engaged and happy.
FinTech Startup: Maintaining security and meeting compliance in a fast-growing, innovative company
One of our clients is a fast-growing FinTech company that provides payroll card solutions for US businesses of all sizes. Their primary product offering is a direct deposit debit card that maximizes direct deposit participation among unbanked employees, eliminating the hassle of cashing paper checks.
Prior to a recent acquisition, the startup was enjoying success as a market leader with a wave of new customer acquisition. Its growth trajectory was also attracting new investors keen to enter the FinTech market. At the same time, the 100-employee company was facing challenges meeting its PCI DSS (Payment Card Industry Data Security Standard) compliance in a rapidly changing regulatory environment. Deep into their growth mode, the company’s leadership was told by investors they could not commit significant funding until new compliances were met.
For FinTech startups, PCI fines can threaten critical cash flow and bottom line profitability. Companies who fail to pass their audits can be fined anywhere from $5k to $100k per month depending on their size. Given their aggressive first-to-market strategy, the pressure was on the team to operationalize solutions and meet compliance immediately.
Originally engaged by a third-party security company to help the company with custom software development, Effectual was introduced by their auditing company to help address its regulatory and security concerns. As an Amazon Web Services (AWS) Advanced Consulting and Well-Architected Partner, Effectual has in-depth experience identifying security vulnerabilities. More importantly, the firm’s core expertise is translating those recommendations into clear, pragmatic steps for operationalizing long-term solutions.
Rapid growth and changing internal roles
As the startup expanded to service its widening customer base, internal roles and operational responsibilities were continually changing. The result was an unclear separation of permissions and duties as well as a lack of capacity or direction for detailed oversight. While former consultants had provided high level recommendations for mitigating security concerns, they had not provided the firm with practical, specific solutions for implementing them, leaving the team uncertain as how to proceed.
Reviewed all seven workloads – particularly related to Primary Account Number (PAN) data – to ensure the company had change management in place. This included security encryption, data storage, and permissions access.
Isolated workloads to keep access separate, creating an Amazon account for each workload.
Outlined clear separation of duties for auditing changes in their environment, with segmented duties and workloads.
Documented and aligned policies, processes, and permissions with internal changes and promotions to provide stability of roles and what tools each will use consistently going forward.
Managing multiple 3rd party vendors and outsourced workloads
The growing company had also become 100% reliant on third-party vendors for its workloads. Keeping eight different vendors informed of its regulatory and compliance requirements and ensuring necessary standards were met had become extremely difficult for the inexperienced team to manage. In addition, the client was at the mercy of its vendors’ competing timelines and unpredictable capacities. This was dramatically slowing its ability to respond to crucial deadlines for compliance. Effectual’s Well-Architected Framework Review quickly surfaced these issues as well as the need for remediation.
Coordinated project management with all third-party vendors to remedy immediate issues affecting compliance.
Built a secure CDE data environment to store PAN data.
Reduced the number of outside vendors to be more manageable and complimentary.
Migrated two PCI-compliant workloads to Amazon using AWS Lambda, Amazon DynamoDB, GuardDuty, and API Gateway.
Outlined plan for migrating remaining workloads to Amazon in the next seven months.
Meeting compliance as an everyday activity
Working with Effectual, the client succeeded in passing its crucial PCI audit in less than 3 months. More importantly, the company has built a DevOps foundation for its future growth and regulatory compliance with everyday operations that ensure its continued success.
As a result, the startup is now skilled at the following:
Understanding its separation of duties, including how many people are involved and needed to facilitate a change in its environment
Documenting and aligning policies, processes, permissions with internal changes and promotions to create greater efficiencies and security
Strategically utilizing third-party vendors and keeping them informed as to its compliance needs
“At first, we brought Effectual on board to build an onboarding web application. But they’ve been far more than just a software development firm. Their DevOps infrastructure expertise, ability to build products in a PCI compliant manner, and emphasis on data security has been a game changer for us.”
Economic Development of Central Oregon (EDCO): Digital solutions that adapt and evolve with a growing organization
EDCO is a regional non-profit that helps companies move, start and grow in Central Oregon.
EDCO had just stepped into a rebranding process to capture its leadership role as an information and networking hub for the region’s business community. At the top of the list: redesigning and overhauling their website. Built years before on a closed, proprietary platform, their non-responsive site didn’t reflect their own progress and was time-consuming for staff to update and keep current.
“The Effectual team has the talent and creativity to respond to whatever you can dream up – so think big.”
– Brian Vierra / Venture Catalyst, EDCO-Bend Venture Conference
Content strategy informs brand messaging
Working closely with tech companies and entrepreneurs, EDCO wanted a digital presence as dynamic as its clients. The group’s new branding aligned with its long-term strategic plan but hadn’t been translated into an effective content strategy that engaged visitors. During an extensive collaboration, we developed the voice and lexicon, user personas, and calls to action that ultimately shaped brand messaging throughout the site as well as in other marketing initiatives.
78% increase – average time on site
30% increase – number of pages viewed
Content strategy process refined overall brand messaging
Discover Your Forest + US Forest Service: Strategic consulting uncovers new opportunities to engage visitors
Discover Your Forest promotes the discovery of Deschutes and Ochoco National Forests and Crooked River National Grassland by enriching the experience of visitors, building community support and creating the next generation of environmental stewards.
DYF’s new leadership team was ready to explore using digital technology to connect visitors and volunteers to its services and expand its donor base. Our discovery process uncovered strategic opportunities for integrating digital solutions that DYF hadn’t considered possible or affordable – launching them into a new phase of innovation and expansion.
Empowering visitors with easy access to information
DYF’s wanted a digital kiosk at the new Cascade Lakes Welcome Center that gave visitors simple access to trail and permitting info. Our team built a custom web app leveraging their existing US Forest Service databases, making trail and use information user-friendly and instantly available to visitors. Directly after launch, the Forest Service began evaluating the web app for regional offices in the Northwest and beyond.
Easy, 24-7 access to visitor information
Increased permitting revenue
New digital solution for Forest Service visitor services
“Partnership with Effectual helped us engage with a wider audience than we’d ever imagined. Their strategic guidance was invaluable and it shows in our final product.”
– Rika Nelson / Executive Director, Discover Your Forest
Mobile app transforms visitor engagement
As conversations evolved, Effectual encouraged DYF to look beyond the web app to a mobile solution that could engage visitors anywhere. The Forest Service had shelved the idea of a mobile app in the past due to cost and technical issues. New research and some collective problem-solving revealed that going mobile was within reach and within budget.
Simple UX makes trail and permit info easy to discover
Expanded engagement and access beyond bricks and mortar experience
Leveraged existing technology platforms with little added cost
Created a standardized, clean set of data deployable across other Forest Service locations
Deep dive business strategy delivers outstanding online experience
Last, the DYF static website needed a complete redesign to boost engagement and connect visitors and volunteers to the group’s mission. Effectual guided their team through an in-depth planning process to identify key personas and calls to action that would drive design and user experience and deliver desired outcomes.
Finely tuned UX development and design aligned with business goals
JHC Technology Services and Infrastructure, an Effectual Company
December 21, 2020
Amazon Web Services (AWS)
Available Products & Services
Cloud Infrastructure as a Service
920-05 – Application, Infrastructure, Hosting and Cloud Computing Services
About Texas DIR
JHC Technology, Inc., an Effectual Company, offers AWS cloud infrastructure services through this contract. Customers can purchase directly through this DIR contract. Contracts may be used by state and local government, public education, other public entities in Texas, as well as public entities outside the state. Resellers are not available for this contract.
How To Order
For product and pricing information, click here or contact Matt Jordan at (814) 421-0617. Reseller Vendors are also available through this contract.
Generate a purchase order made payable to JHC Technology, Inc. You must reference the DIR Contract Number DIR-TSO-4064 on your purchase order.
E-mail or fax your purchase order and quote form to your designated vendor or reseller sales representative.
Chief Information Officer–Solutions and Partners 3 (CIO-SP3)
Small Business Government-Wide Acquisition Contract (GWAC)
Small Business Ramp On
About CIO-SP3 Small Business
JHC Technology, Inc., an Effectual Company, was awarded the CIO-SP3 Small Business contract; group SB. The Chief Information Officer–Solutions and Partners 3 (CIO-SP3) Small Business Government-Wide Acquisition Contract (GWAC) is a ten (10) year Indefinite Delivery/Indefinite Quantity (IDIQ) contract. This contract is intended to provide information technology (IT) solutions and services as defined in FAR 2.101(b) and further clarified in the Clinger-Cohen Act of 1996. These IT solutions and services include, but are not limited to, health and biomedical-related IT services to meet scientific, health, administrative, operational, managerial, and information management requirements. The contract also contains general IT services partly because medical systems are increasingly integrated within a broader IT architecture, requiring a systems approach to their implementation and a sound infrastructure for their operation.
Task Area 1: IT Services for Biomedical Research, Health Sciences, and Healthcare
Task Area 2: Chief Information Officer (CIO) Support
Task Area 4: Outsourcing
Task Area 5: IT Operations and Maintenance
Task Area 6: Integration Services
Task Area 7: Critical Infrastructure Protection and Information Assurance
App Modernization: Strategic Leverage for Managing Rapid Change
The last few months of the COVID crisis have made this even more evident, dramatically exposing security faults and the limitations of outdated monolithic applications and costly on-premises infrastructure. This lack of modernization is preventing many businesses and agencies from adapting to new economic realities and finding a clear path forward.
Whether improving efficiencies with a backend process or creating business value with a new customer-facing app, modernizing your IT solutions helps you respond quickly to changing conditions, reduce your compliance risk, and optimize costs to match your needs. Applications that are already architected to take advantage of the cloud also provide flexibility to address scalability and performance challenges as well as to explore new opportunities without disrupting budgets and requiring heavy investment.
First, what defines technologies that are NOT modern?
Inflexible monolithic architectures
Inability to scale up or down with changes in demand
Security only implemented on the outside layer, not at the component layer
Costly on-premises infrastructure
Legacy hardware burdens
Waterfall development approaches
Maintaining legacy technologies is more expensive than modernizing them
Some of the most striking examples of the complexity, costs, and failures associated with legacy technologies have recently been seen in the public sector. In fact, some state unemployment systems have failed to handle the overwhelming increase in traffic and demand, impacting those in greatest need of assistance. There are those that are already taking measures within the public sector. Beth Cappello, acting CIO of the US Department of Homeland Security, recently stated that had her predecessors not taken steps to modernize their infrastructure and adopt cloud technologies, the ability for DHS personnel to remain connected during the pandemic would have been severely impacted.
Many government applications run on 30+ year-old mainframe computers using an antiquated programming language, creating a desperate need for COBOL developers to fix the crippled technologies. What the situation reveals is the dire need to replatform, refactor, and rearchitect these environments to take advantage of the scalability, reliability, and performance of the cloud.
Benefits of modernization:
Security by design
Resilient microservices architecture
Automated CI/CD pipeline
Infrastructure as code
Rapid development, increased pace of innovation
Better response to customer feedback and market demands
Flexible, pay-as-you-go pricing models
Automated DevOps processes
Scalable managed services (ie: Serverless)
In-depth analytics and data insights
The realities of preparing for the unknown
As a result of shelter-in-place orders since early March, we have seen both the success of customers who have modernized as well as the struggles of those still in the process of migrating to the cloud.
Food for All is a customer with a farm-to-table grocery app that experienced a 400x increase in revenue as people rushed to sign up for their service during the first few weeks of the pandemic. Because we had already built their architecture for the Amazon Web Services (AWS) cloud, the company’s technology environment was able to scale easily to meet demand. In addition, they have a reliable DevOps environment that allowed them to immediately onboard more developers to begin building and publishing new features based on user feedback.
Unfortunately, other customers have not been able to adapt as quickly.
When one of our retail clients lost a large number of customers in the wake of COVID, they needed help scaling down their environment as rapidly as possible to cut their costs on AWS. However, the inherited architecture had been written almost 10 years ago, making it expensive and painfully time-consuming to implement adjustments or changes. As a result, the company is currently weighing whether to turn off their app and lose revenue or invest in modernizing it to recover their customers.
For another large enterprise customer, the need to reduce technology costs meant laying off a third of their payroll. Though our team is helping them make progress on refactoring their AWS workloads, they were still unable to scale down 90% of their applications in time to avoid such a difficult decision. The situation has significantly increased their urgency to modernize.
The need for a cloud-first modernization service provider
With AWS now 14 years old, it is important to realize that modernization is just as important to early adopters as it is for the public sector’s legacy workloads. In fact, many early cloud adopters have not revisited their initial architectures to ensure they are taking advantage of the hundreds of new features and services released by AWS each year (during Andy Jassy’s 2019 re:Invent keynote alone, he announced 30 new capabilities in 3 hours). For these reasons, and many more, our Modernization Engineers help customers make regular assessments of their cloud infrastructure and workloads to maintain a forward-looking, modern IT estate.
Whether migrating out of an on-premise data center or colo, rearchitecting an existing cloud workload, or developing with new cloud-native features,it has never been more important to implement a modern cloud strategy. This is particularly true for optimizing services across your organization and embracing security as a core pillar.
According to Gartner, 99% of cloud security failures through 2025 will be the customer’s fault. Clearly, no organization wants to be a part of this statistic. Ongoing management of your critical workloads is a worthy investment that ensures your mission-critical assets are secure. The truth is that if security isn’t done right, it simply doesn’t matter.
We work frequently with customers looking to completely exit their data center infrastructure and migrate to an OPEX model in the cloud. In these engagements, we identify risks and dependencies using a staged approach to ensure the integrity of data and functionality of applications. However, this migration or “evacuation” is not an end state. In fact, it is often the first major milestone on a client’s journey toward continuous improvement and optimization. It is also nearly impossible to do efficiently without modern technology and the cloud.
Modern cloud management mitigates risk and enables modernization
While some workloads and applications may be considered cloud-ready for a relatively straightforward lift and shift migration, they can usually benefit from refactoring, rearchitecting, or replatforming based on a thorough assessment of usage patterns. Cloud adoption on its own will only go so far to improve performance and organizational flexibility.
A modern digital strategy allows you to unlock the true capabilities of the cloud, increasing scalability, agility, efficiency, and one of the most critical benefits of any modernization initiative – improved security. Modernized technologies can also utilize cutting edge security protocols and continuous compliance tools that are simply not available with physical infrastructure.
Unlike traditional MSPs (Managed Service Providers) who manage on-premises servers in physical data centers, Effectual is a cloud-first ModernizationService Provider that understands how to modernize applications, their metrics, operational costs, security implications, and compliance requirements. When our development team finishes a project, our customers can Cloud Confidently™ knowing that their environment is in experienced hands for ongoing management.
Most importantly, the path to modernization is not necessarily linear, whether you are developing an application specifically for the cloud, refactoring or rearchitecting as part of a data center migration, or updating and securing an existing cloud environment. New ideas, priorities, and changes to the world we live in require that we adapt, innovate, and rethink our approach to solving business challenges in even the most uncertain times.
When your organization or team needs the power to pivot, we have the Modernization Engineers, systems, tools, and processes to support that change.
Not so long ago, R&D departments had to fight for internal resources and justify capital expenditures in order to explore new technologies. Developing on premise solutions was expensive and time-consuming, and decisions were focused on ensuring success and avoiding failure.
In the past 5 years, cloud platforms have radically sped up the pace of innovation, offering companies of all sizes the ability to build, test, and scale solutions at minimal cost. Technology is now a tool to differentiate yourself from your competitors, increase your margins, and open up new markets.
Small investments, big payoffs
By committing only a small portion of your budget to R&D, you can now leverage plug and play cloud services to experiment and test Proofs of Concept (POCs) with potentially huge bottom line payoffs. For large companies, utilizing POCs requires a shift away from risk-averse waterfall development to an agile approach that embraces failure as a path to innovation.
Enterprise organizations can learn from entrepreneurs, who’ve been natural first adopters when it comes to cloud solutions. Startups aren’t afraid of using pay-as-you-go services to build quick POCs for validating markets, testing technical features, and collecting customer feedback. Far more comfortable with agile development, successful early stage companies like Effectual customer Warm Welcome are adept at taking calculated risks and viewing failure as an invitation for learning.
In contrast, enterprise customers may struggle at first to embrace an agile approach and accept failure as an opportunity for insight. As established businesses, they also make the mistake of assuming reputation alone will ensure successful outcomes and often downplay the importance of customer feedback. However, this changes quickly after companies gain experience with POCs and understand the value of testing their assumptions before committing to building out final solutions.
POC vs MVP: What’s the difference?
A Proof of Concept is the first phase of designing a software application. A POC allows you to quickly solve a business challenge for a specific use case in order to:
Test technical functionality
Collect user feedback
Determine market acceptance
POCs are timed-boxed (defined by # of hours), with clear KPIs (key performance indicators) for measuring your results. This keeps costs low and provides rapid insights into what changes need to be made before you invest significant resources to scale it.
POCs are rolled out to a controlled, focused group of users (“friends and family”) with the goal of quickly figuring out cost and technical issues. It’s not uncommon to go through 3-4 POCs before finding the one you’re ready to advance. Failure is an accepted and necessary part of this process.
For example, one of our large retail customers has dedicated $4k/month to its backlog pipeline for R&D. At the beginning of the year, we sat down with their team to identify 4-5 business problems the company wanted to tackle. For one particular POC, we developed and tested two different POCs (one cloud-based, one on-premise) before finding a hybrid solution that was the right compromise between cost and functionality.
To minimize risk, they rolled out their hybrid POC to a single store location in order to collect user feedback. Only after making recommended changes did the company commit to moving forward with an MVP at multiple locations across several states. Within 18 months, they have seen a significant return on their investment in both higher sales and increased customer retention.
A Minimum Viable Product (MVP) is a featured-boxed solution that turns your proven concept into a functional basic product you can test with a wider user base. While it resides outside of a critical business path, an MVP usually requires greater investment and takes longer to evaluate. The goal of an MVP is to:
Increase speed to market
Establish loyal users
Prove market demand
Collect broader customer feedback
Organizations of any size can use POCs to ensure the fast delivery of a final product that meets the needs of customers and provides a measurable return on investment. Learn more about how a POC can drive your business forward.
Have an idea or project to discuss? Contact us to learn more.
AWS IoT Solutions Accelerate the Potential of Edge Computing
IoT is revolutionizing consumer markets, large-scale manufacturing and industrial applications at an incredible pace. In virtually every industry, these technologies are becoming synonymous with a competitive advantage and winning corporate strategies.
We’re witnessing the same trend with our own customers, as companies integrate IoT solutions with their offerings and deploy edge devices to improve customer experience, reduce costs, and expand opportunities.
Installing these smart devices and collecting data is relatively easy. However, processing, storing, analyzing, and protecting massive volumes of that data is where (Internet of) Things gets complicated.
As an AWS Premier Consulting Partner, Effectual guides customers on how to leverage Amazon Web Services (AWS) innovation for their IoT solutions. This includes building performant, cost-effective cloud architecture based on the 5 Pillars of the Well-Architected Framework that scale quickly and securely process real-time streaming data. Most importantly, we apply AI and machine learning (ML) to provide clients with meaningful analytics that drive informed business decisions.
AWS Lambda: Serverless computing for continuous scaling A fully-managed platform, AWS Lambda runs code for your applications or backend services without requiring any server management or administration. It also scales automatically to workloads with a flexible consumption model where you pay for only the computing resources you consume. While Lambda is an excellent environment for any kind of rapid, scalable development, it’s ideal for startups and growing companies who need to conserve resources while scaling to meet demand.
We successfully deployed AWS Lambda for a project with Oregon-based startup Wingo IoT. With an expanding pipeline of industrial customers and limited resources, Wingo needed a cost-efficient, flexible architecture for instant monitoring and powerful analytics. We used Lambda to build a custom NOC dashboard with a comprehensive view of real-time operations.
DynamoDB: Fast access with built-in security We use DynamoDB with AWS Services such as Amazon Kinesis and AWS Lambda to build key-value and document databases with unlimited capacity. Offering low latency and high performance at scale, DynamoDB can support over 10 trillion requests a day with secure backup and restore capabilities.
When Effectual client Dialsmith needed a transactional database to handle the thousands of records per second created by its video survey tool, we used DynamoDB to solve its capacity constraints. The service also provided critical backup and restore capabilities for protecting its sensitive data.
In our experience, AWS IoT solutions both anchor and accelerate the potential of edge computing. Services such as AWS Lambda and Amazon DynamoDB can have a lasting, positive impact on your ability to scale profitability. Before you deploy IoT technologies or expand your fleet of devices, we recommend a thorough evaluation of the cloud-based tools and services available to support your growth.
Have an idea or project to discuss? Contact us to learn more.
5 Reasons Your Development Team Should be Using the Well-Architected Framework
Amazon Web Services (AWS) offers the most powerful platforms and innovative cloud technologies in the industry, helping you scale your business on demand, maximize efficiencies, minimize costs, and secure data. But in order to take full advantage of what AWS has to offer, you need to start with a clear understanding of your workload in the cloud.
How to Build Better Workloads Whether you’re working with an internal team or an outsourced consulting partner, the AWS Well-Architected Framework is an educational tool that builds awareness of steps and best practices for architecting for the AWS Cloud. We begin all of our development projects with a Well-Architected Review to give clients full visibility into their workload. This precise, comprehensive process provides them essential insights for comparing strategies, evaluating options, and making informed decisions that add business value. Based on our experience, using well-architected best practices and design principles helps you:
1 – Plan for failure One of the primary Well-Architected design principles is to architect for failure. This means knowing how to mitigate risks, eliminate downtime, prevent data loss, and protect against security threats. The Well-Architected process uncovers potential security and reliability vulnerabilities long before they happen so you can either avoid them or build a plan proactively for how you’ll respond if they do. This upfront effort can save you considerable time and resources. For example, having a disaster recovery plan in place can make it far easier for you to spin up another environment if something crashes.
Clients who plan for failure shrink their Recovery Time Objective (downtime) and their Recovery Point Objective (data loss) by 2000%.
2 – Minimize surprises Mitigating your risks also means minimizing surprises. The Well-Architected Framework offers an in-depth and comprehensive process for analyzing your choices and options as well as for evaluating how a given decision can impact your business. In our Well-Architected Reviews, we walk you through in-depth questions about your workload to create an accurate and holistic view of what lies ahead. When the review answers and recommendations are shared with all departments and stakeholders of a workload, they’re often surprised by the impacts of decisions on costs, performance, reliability and security.
3 – Understand the trade-offs of your decisions Building well-architected workloads ensures you have options for responding to changing business requirements or external issues, with a structure for evaluating the trade-offs of every one of those choices. If you feel your application isn’t performant, you may have 10 different possible solutions for improving performance. Each one has a tradeoff, whether it be cost, maintainability, or more. The Well-Architected Framework can help your team decide the best option.
Identifying and executing refactoring options based on modern technologies and services can save up to 60% of architecture costs.
As an organization, you should never feel boxed in when it comes to options for improving your workload. The process and questions presented in the Well-Architected Framework can help both your technical and business departments look at all options and identify which ones will have the most positive business impact.
In 75% of the workloads we encounter, the technology department is making the decisions, which means there is no input from business stakeholders as to impacts.
4 – Develop KPIs to monitor the overall health of your application Choosing KPIs that integrate both technical and business indicators gives you valuable insights into your application’s health and performance. With a Well-Architected approach, you can automate monitoring and set up alarms to notify you of any deviance from expected performance. Once you’ve established this baseline, you can start exploring ways to improve your workload.
KPI’s should be driven by business and should include all areas of your organization including Security, Finance, Operations, IT and Sales. The Well-Architected Framework provides a well-rounded perspective of workload health.
After a Well-Architected Review, it’s common to have 90% of KPIs defining the health of your application come from other business departments – not just from the IT team.
5 – Align your business requirements with engineering goals Following well-architected best practices also facilitates a DevOps approach that fosters close collaboration between business managers and engineers. When the two are communicating effectively, they understand both the engineering concepts and the business impacts of their decisions. This saves time and resources and leads to more holistic solutions.
To fully leverage the AWS Cloud, make sure your development team has a strong foundation in the Well-Architected Framework. You’ll be able to build better workloads for the cloud that can take your business further, faster.
Have an idea or project to discuss? Contact us to learn more.
Enabling IT Modernization with VMware Cloud on AWS
Cloud and virtualization technologies offer a broad range of platform and infrastructure options to help organizations address their operational needs, no matter how complex or unique, and reduce their dependence on traditional data centers.
As the demand for cloud and cloud-compatible services continues to grow across departments within organizations, cloud adoption rates are steadily rising and IT decision makers are realizing that they no longer need to be solely reliant on physical data centers. This has led countless organizations to shrink their data center footprints.
The benefits unlocked by VMC on AWS can have significant impacts on your organization…including the impressive performance of a VMware environment sitting on top of the AWS backbone.
VMware Cloud on AWS is unique in bridging this gap, as it utilizes the same skill sets many organizations have in-house to manage their existing VMware environments. Sure, there are considerations when migrating, but ultimately the biggest change in moving to VMware Cloud (VMC) on AWS is the underlying location of the software defined data center (SDDC) within vCenter. The benefits unlocked by VMC on AWS can have significant impacts on your organization – eliminating the need to worry about the security and maintenance of physical infrastructure (and the associated hands on hardware to address device failure) as well as the impressive performance of a VMware environment sitting on top of the AWS backbone.
Technology That Suits Your Needs
Full and partial data center evacuations are becoming increasingly common and, while there are instances of repatriation (organizations moving workloads from the cloud back to the data center), the majority of organizations are sticking with “cloud-first” policies to gain and maintain business agility. Sometimes, however, even a company that’s begun their IT modernization efforts may still have systems and applications hosted on-premises or in a data center.
This may seem to indicate some hesitance to fully adopt the cloud, but it’s usually due to long-term strategy, technical barriers to native cloud adoption, or misconceptions about cloud security and compliance requirements. It’s rare to find an organization that isn’t loaded with technical debt, fully committed to specific software, tied to lengthy data center commitments – or all of the above.
Mission-critical legacy applications may not be compatible with the cloud, and organizations lack the resources or expertise to refactor those applications so that they can properly function in a native cloud environment. Or perhaps there’s a long-term digital strategy to eventually move all systems and applications to the cloud but, in the meantime, they’re still leashed to the data center. Scenarios like these, and many more, are ideal for VMware Cloud on AWS, which allows organizations to easily migrate legacy VMware workloads with minimal refactoring or rearchitecting, or extend their existing data center systems to the cloud.
New, But Familiar
VMware Cloud on AWS was developed in collaboration between VMware, a pioneer and global leader in server virtualization, and AWS, the leading public cloud provider, to seamlessly extend on-premises vSphere environments to SDDCs built on AWS. VMC on AWS makes it easier for organizations to begin or expand their public cloud adoption by enabling lift and shift migration capabilities for applications running in the data center or on-premises VMware environments.
VMC on AWS also has a relatively minimal learning curve for in-house operations staff because, despite being hosted on AWS, it’s still VMware vSphere at its core and the environments are managed using the vCenter management console. This familiar toolset allows IT teams to begin utilizing the cloud without any major workforce retraining and upskilling initiatives because they can still use VMware’s suite of server virtualization and management tools.
The Right Tools for the Job
The vSphere suite of server virtualization products and vCenter management console may be familiar, but they’re far from outdated or limited. VMware continues to invest in the future, strengthening its cloud and virtualization portfolio by enhancing their existing offerings and developing additional services and tools to further enable IT modernization and data center evacuations.
These efforts mean we can expect VMware to continue playing a major role in helping organizations achieve and maintain agility by ensuring secure workload mobility across platforms, from public cloud to private cloud to hardware.
HCX, which essentially consists of a series of integrations that establish connectivity across systems and platforms and allows workloads to be migrated without any code or configuration changes, is regularly updated to enhance its functionality. VMware HCX can be used to perform live migrations using vMotion and bulk migration for up to 100 VMs at a time. VMware HCX can also provide a secure, accelerated network extension which, beyond providing a seamless migration experience and minimizing operational impacts usually associated with migrating workloads, helps improve the environment’s resiliency through workload rebalancing. This same functionality plays a critical role in disaster recovery and business continuity by replicating data across multiple locations.
A Thoughtful Approach to Modernization
Whether an organization is prioritizing the optimization of spend, revenue growth, streamlining operations, or revitalizing and engaging their workforce, a mature and robust digital strategy should be at the heart of the “how.” Cloud adoption will not solve these business challenges on its own – that requires forethought, planning, and expertise.
It can be challenging to make the right determinations about what’s best for your own unique business needs without a clear understanding of those needs. And for organizations still relying on old school hardware-based systems, the decision to remain with on-premises deployments, move to the cloud, or lift and shift to a platform like VMC on AWS requires a comprehensive assessment of their applications, hardware, and any existing data center/real estate commitments.
Internal teams may not have the specific technical expertise, experience, or availability to develop suitable digital strategies or perform effective assessments, especially as they focus on their primary day to day responsibilities. As an AWS Premier Consulting Partner with the VMware Master Services Competency in VMware Cloud on AWS, Effectual has established its expertise in VMware Cloud on AWS, making us an ideal partner to help ease that burden.
Cloud adoption doesn’t happen overnight, and organizations have to ensure disparate technologies, which may be at very different stages of their respective lifecycles, mesh well. They need to develop an appropriate modernization strategy and determine the best fit for each application and workload. The right partner can play a critical role in successfully overcoming these challenges.
FISMA Moderate Requirements met with AWS Infrastructure
Effectual led a Federal Government client in their journey from on-premises infrastructure to a secure cloud environment in AWS.
This Federal Government customer required a move from its on-premises infrastructure to a centralized cloud environment. This move was predicated on the requirement for increased security, flexibility in provisioning infrastructure, and a refresh of technology. The new AWS infrastructure must also be assessed at a FISMA Moderate level for production.
Our team led the discovery, architecture, and implementation of an agency’s new infrastructure. We designed a multi-region, international architecture that allowed end users to quickly access virtual desktops at regions closest to those users. The centralized management and region-based architecture allowed devices to move outside the boundary, the virtual desktops infrastructure scaled as users joined around the world, and the agency was able to provision lower cost technology, such as thin clients, to achieve a refresh.
Our team supported the agency in its ATO efforts by provisioning compliant infrastructure and services in alignment with FISMA Moderate controls, then produced documentation supporting the architecture, allowing the agency to get a full ATO.
Our AWS-based architecture and deployment supported configuration of infrastructure to meet minimum workloads, which then scaled as users came online. Additionally, multiple user desktops could be provisioned on a single server, cutting down on associated costs.
The agency’s network needed to be overhauled as a result of security concerns. With the AWS backbone and multi-region architecture, users experienced a decrease in latency and the zero-trust model improved network security.
Satellite Imagery Analysis Simplified with Serverless Infrastructure
Effectual worked with a Federal Government customer to provide a mission critical solution that simplified its Land Satellite sensor processing software of the Earth’s land surface.
The images provide uninterrupted data to help land managers and policymakers make informed decisions about our natural resources and the environment.
A Federal Government customer looked to us to migrate its on-premises infrastructure to a Serverless infrastructure in AWS to ensure cost optimization, availability, and application performance while logging satellite images.
Our team implemented AWS Lambda, AWS Batch, Kubernetes, and Amazon EKS. This ensured the client’s ability to collect satellite images that would be used to help scientists track land change due to climate, urbanization, drought, wildfire, and biomass changes.
We implemented AWS Lambda to run code without servers. By implementing Serverless infrastructure the client was able to reduce cost by 80%.
Our team implemented Kubernetes to provide automated container orchestration and higher availability across multiple regions. This allowed users – both domestic and abroad – to access satellite photos more efficiently via the web for personal and private use.
We set up serverless storage to compact the client’s satellite imagery retrieval process from 2 weeks to 2 hours.
Service Employees International Union (SEIU) Application Migration
When flooding took out the New York data center of the national nonprofit, SEIU, the organization found a need to act on a move to the AWS cloud.
Through third-party and cloud-native tools, we provided the infrastructure, resources, and products necessary to efficiently migrate workloads.
The national nonprofit serves branches of the organization with centralized IT based out of its New York offices. When NYC was hit by Hurricane Sandy in 2012, it led to flooding of the organization’s data center, housed in the basement of the building. The resulting outage took a week to recover from. The nonprofit needed a cloud-based backup solution to ensure that it could be prepared against future disasters.
We began with an assessment of the organization’s data center posture, then created a migration plan and proposed architecture to support the nonprofit moving forward in AWS. We configured VPCs, subnets, networking, and configured access policies. We also connected a third-party disaster recovery service to ensure consistent synching of information between on-premises and cloud servers.
Piece of Mind
After going without its critical IT infrastructure for a week, the nonprofit had confidence its cloud infrastructure would be highly available.
The AWS infrastructure included VPN connectivity to the on-premises network in order to replicate Active Directory and SQL databases to ensure ongoing operations.
In addition to an initial VPN connection, our team configured remote VPN connectivity from field offices in seven east coast cities to ensure all users could access the environment in the event of a failure.
GenomeNext is a genomic informatics company dedicated to accelerating the promise and capability of predictive medicine and scientific discovery. It commercializes genomic analysis tools and integrated systems for the evaluation of genetic variation and function.
The advanced informatics and data management solutions are designed to simplify, expedite and enhance genetic analysis workflows. GenomeNext solutions provide the market with genomic data and analysis at an unprecedented combination of performance, quality, cost and scale without requiring the investment in high-performance computing resources and specialized personnel. The proprietary platforms address a broad range of highly interconnected markets, including sequencing, genotyping, gene expression, and molecular diagnostics. GenomeNext customers include leading genomic research centers, academic institutions, government laboratories, and clinical research organizations, as well as pharmaceutical, biotechnology, agrigenomics, and consumer genomics companies.
GenomeNext needed a more efficient way to develop and deploy application changes to its Amazon Web Services Genomics Cloud Platform while maintaining high level of security and compliance.
We worked with GenomeNext to design efficient development and agile management process, setup internal DevOps software and AWS infrastructure components, mapped processes to appropriate security and compliance controls, integrated third party DevOps tools with the GenomeNext Cloud platform, implemented development life cycle environments (Dev, QA, and Prod) on AWS, monitored and reduced AWS costs, and architecture high availability and disaster recovery. Our solution enhanced GenomeNext’s ability to quickly and securely roll out application development and infrastructure changes with minimal to zero downtime through the use of tools such as AWS Elastic Load Balancing, AWS CloudWatch, AWS CloudFormation, and AWS CodeDeploy.
GenomeNext recognized the advantages of DevOps automation by a significant increase in deployment frequencies, a dramatic decrease in deployment failures, immediate recovery of failed deployments, and reduction in the time required for changes.
By combining AWS and DevOps, GenomeNext can automate the deployment of an exact copy of its Production solution within minutes into any AWS region, allowing it to meet its recovery time objectives.
GenomeNext realized cost saving utilizing DevOps and AWS. Cost saving came in terms of maintaining a small staff, increased quality of products, reduction deployment complexity, and faster time to market.
Supporting the Delivery of Early Warning Signs for Earthquakes
Effectual delivered a mission-critical solution to a Federal Government Client that ensured the delivery of early warning alert notifications for earthquakes and other natural disasters over multiple geographical locations to save lives.
This could not have been done without a Cloud-based solution to ensure a resilient system.
This Federal Government customer required a move from its on-premises infrastructure to a centralized Cloud environment. The client looked to our team to handle high availability architecture and fault tolerance to meet workloads over multiple geographical locations quickly after a natural disaster. The solution required improved resilience and redundancy capabilities, application performance, and control monitoring.
Our team built out a highly available and scalable infrastructure to meet demand in the wake of a disaster. We utilized the customer’s containerized solution and created a pipeline leveraging a GitLab Runner in Amazon Web Services (AWS) to manipulate and manage the AWS Elastic Kubernetes Service (EKS) deployments. This ensured the client’s ability to deliver early warnings for natural disasters through their application.
Our team configured Amazon CloudWatch metrics to identify a surge in traffic in the event of a disaster. This fully integrated AWS service is built with more resilience. Kubernetes was implemented to provide automated container orchestration and higher availability to reach across multiple regions.
We created a proprietary AWS-hosted Git solution to do all the linking, testing, and delivery to code. Our solution increased the rate at which the client released updates to the solution by 90%.
We deployed a GitLab Runner in conjunction with GitLab Continuous Integration to ensure all applications were provisioned through a pipeline. These necessary changes led to extreme version control and expediting developer updates.
Predictive Analytics: Volcanic Activity Analyzed Through Moving Magma
Effectual delivered a mission-critical solution to a federal government client that ensured their sensor processing software was able to predict volcanic activity through moving magma.
This information is used to help scientists forecast seismic activity over multiple geographical locations. This could not have been done without a Cloud-based solution to ensure a resilient system.
Our customer required a move from its on-premises infrastructure to a centralized Cloud environment in AWS. They looked to our team to handle high availability architecture and fault tolerance to meet workloads over many geographical locations quickly after a natural disaster.
We provided a highly available and scalable infrastructure that ensured efficiency in wake of volcanos and other natural disasters. This sensor processing solution ensured predictive analytics, resilience, and scalability.
We worked with the customer to create a solution that ensured the user could collect volcano data to analyze and utilize for machine learning to better predict when volcanoes erupt.
Our team configured Amazon CloudWatch metrics to identify a surge in traffic in the event of a disaster. Kubernetes was implemented to provide automated container orchestration and higher availability to reach across multiple regions.
We configured EC2 instances that ensure adequate capacity to meet traffic demands and compute capacity. Our team automated launch configurations to allow the client to quickly launch and/or scale application servers in target environments in the future.
Bird Conservation Science Enabled by Automated Monitoring and Analysis of Migration Patterns
Effectual led a Federal Government client in need of automation, reliability, and efficiency for their bird identification website.
The customer supports the collection, archiving, management and dissemination of information from banded and marked birds in North America. This information is used to monitor the status and trends of resident and migratory bird populations.
This Federal Government customer required a move from its on-premises infrastructure to a centralized cloud environment. The client looked to our team to redesign their website, creating a system that would produce automated checks to save time and manual effort when registering banded and checked birds into the database.
Our team assisted the customer in creating a system that would require minimal effort to keep up and running for years. This system saved time and manual effort through the implementation of Amazon Elastic Compute Cloud (Amazon EC2) to automate cron jobs for repetitive tasks to push all submitted web surveys from bird hunters and enthusiasts to the on-premises database. When banded birds were checked in, the system would be able to ensure the identification was correct, eliminating the need to manually check that information.
We utilized Amazon EC2 to automate database syncing. This allowed the bird banding lab to be more efficient when a bird was reported on their website. The client no longer needed to manually log and input the bird species. AWS CloudFormation was implemented to reduce manual work while developing an environment, ensuring productivity when debugging issues.
We used GitLab Continuous Integration in conjunction with GitLab Continuous Deployment to check code for errors, expediting developer changes.
Our team implemented Amazon CloudWatch Events for serverless workflow to trigger Lambda functions. Without having to provision or manage, the client was able to keep the same server running by keeping it warm with a CloudWatch Event. This reduced response times from 3 seconds to a couple hundred milliseconds.
Ensuring Least Privilege Access: Implementing an Active Directory Federation Service
Effectual led the implementation of an enterprise grade Active Directory Federation Service (ADFS) for a large Federal Government client.
Effectual enabled reliable and secure cyberspace capability by providing a highly innovative network architecture, engineering, integration, and simulation services with unrivaled expertise and commitment.
The client looked to our team to move its highly disparate environment into a highly collaborative one. By implementing Federated Access to the Amazon Web Services environment, this ensured least privilege access to client users.
We worked with the client to setup an AWS Identity and Access Management (IAM), federated sign-in through Active Directory (AD), and Active Directory Federation Services (ADFS). This ensured least privilege access to client users.
Our team enabled reliable collaborative connectivity to a cadre of remote workers that needed access to the system while utilizing the ADFS PIV card solution.
We were able to meet all security requirements by using a federated solution, allowing the client to set permissions and access levels across different systems. The Federated solution also improved auditing management of credentials.
We implemented AWS CloudFormation to create a template to use when multiple accounts register in the system. This led to an increase in efficiency and ensures consistent configurations overtime.
Members of the engineering team had the opportunity to attend Amazon Web Services’ annual re:Invent conference in Las Vegas.
Every year, AWS announces dozens of customer-sought features at the event (and some leading up to the event in what the community has dubbed “pre:Invent”). In this blog- a second in a series of two on re:Invent – we’ll touch on new announcements from this year’s conference:
Amazon excited data scientists with the announcement of Amazon SageMaker Studio which provides an easier experience for building, training, debugging, deploying and monitoring machine learning models with an integrated development environment (IDE).
Amazon Athena federated queries turn almost any data source into a query-able data repository, opening opportunities to gather insights based on data from many different sources in different formats.
Amazon Detective makes it easy to analyze, investigate, and quickly identify the root cause of potential security issues or suspicious activities by using machine learning, statistical analysis, and graph theory.
Automate code reviews with Amazon CodeGuru, a machine learning service which helps development teams identify the most expensive lines of code in their applications and receive intelligent recommendations on how to fix or improve their code.
Amazon Simple Storage Service (S3) adds additional security measures and flexibility to share data with others by introducing Amazon S3 Access Points.
With all the new features coming out of re:Invent, it was difficult for us to pick our top picks, but our team is quickly becoming experts in all the new features and already utilizing them in delivering first-class cloud infrastructure to our clients.
Members of the engineering team had the opportunity to attend Amazon Web Services’ annual re:Invent conference in Las Vegas.
Every year, AWS announces dozens of customer-sought features at the event (and some leading up to the event in what the community has dubbed “pre:Invent”). This blog is the first in a two-part series related to re:Invent announcements from the 2019 conference:
A feature requested by customers since Amazon Elastic Kubernetes Service (EKS) was announced last year, AWS Fargate support for Amazon Elastic Kubernetes Service will revolutionize the way organizations use the popular Kubernetes container management tools in the cloud and radically reduce the maintenance required for running Kubernetes on AWS.
A pre:Invent announcement that you might have missed if you blinked, CloudFormation Registry and third-party resource support adds the ability to manage virtually any third-party application resource using CloudFormation, an infrastructure as code tool helping organizations iterate faster with repeatable cloud resource definitions stored as code.
Andy Jassy rocked the re:Invent stage in 2018 when he announced AWS Outposts, a new offering to take AWS’ computing capacity into your own data center. This service was made available in 2019, opening a wealth of potential for applications which need to stay local for regulatory or performance purposes.
The Amazon Builder’s Library is a curated list of content written by Amazon’s own technical leaders to illustrate how Amazon builds world-class services and infrastructure.
With all the new features coming out of re:Invent, it was difficult for us to pick our top picks, but our team is quickly becoming experts in all the new features and already utilizing them in delivering first-class cloud infrastructure to our clients.
Real-time logging of Tsunami Data Aids in Disaster Response
Effectual worked with a federal government customer to provide information for local land-use and emergency response planning to avoid development in hazardous zones and to plan evacuation routes to communities along low-lying coastlines vulnerable to tsunamis.
The customer looked to our team to quickly and effectively move their public-facing web applications and internal applications to the AWS cloud to ensure resiliency, availability, and real time logging of tsunamis.
We implemented a solution comprised of Amazon CloudWatch, AWS CloudTrail, Alarms, and Serverless Storage. This ensured the clients ability to collect data to help scientists understand tsunamis through their application to develop how to most effectively improve preparedness and response to tsunamis.
Resiliency We implemented Amazon CloudWatch to schedule data collection that self-triggers when a tsunami is detected.
Availability By implementing AWS CloudTrail the client was able to easily access tsunami data to help scientists understand the sources of local tsunamis so that the impacts of future events may be mitigated.
Real Time Logging Our team set up serverless storage to collect data from these seismic networks to process key components in the impact of tsunamis.
As cloud computing continues to grow within the State and Local Government industry, it has become increasingly popularized in the Education industry.
AWS started an initiative called AWS Educate to provide students and educators with the training and resources needed for cloud-related learning. Cloud computing skills are in high demand throughout the state of Texas, especially as an increasing number of state and local government agencies are embarking on migrating to the cloud. It has been a slow process for the government to migrate to the cloud, but the education sector is ahead of the process. This is due to high demand for the students, teachers, faculty, staff, and parents needing access to critical information using any devices from anywhere. Educators can benefit by migrating to the cloud: it’s cost efficient, offers stable data storage, development and test environments, easier collaboration for users, enhanced security without add-on applications, simple application hosting, minimizes resource costs, and fast implementation and time-to-value.
With all the capabilities of Cloud environments, the Education industry still has a long way to go. There are certain school districts and even Higher Education institutions, that do not have the amount of access as some of their counterparts. Cloud vendors could make a difference and solidify cloud adoption by offering Cloud education to urban neighborhood schools with laptops, computers, and access to training and certifications. As a start, the three major Cloud providers offer cloud education assistance to students:
When it comes to the rapid advancement in the IT industry, I encourage other young minorities, including my daughter, to pursue a career in the technology industry. Children are the future and Cloud platforms will be the leading solution across all markets.
We offer a bundled package for new users which includes an assessment of their current infrastructure which can be beneficial to any Higher Education Institution or K-12 organization. We can build the future together and keep rising to greater heights!
Reach out to Thy Williams, [email protected], to learn more about our capabilities and discuss our starter package.
Pharmaceutical companies have long been familiar with the pros and cons of cloud vs. non-cloud environments. The same discussions took place when companies in other industries began transitioning from on-premises to outsourced providers.
However, the pharmaceutical industry, and the data they manage, is within the scope of the Federal Drug Administration (FDA). With the FDA’s purview increasing over the years, and the globalization of their compliance oversight (now including over 150 countries exporting FDA‑regulated products to the United States), they have put more of the onus of following regulations on the pharmaceutical companies.
Enhancing Security Through Strategy and Architecture
In response to the FDA asking questions regarding the Code of Federal Regulations Title 21, Part 11 (CFR 21 Part 11), companies complying with GxP regulations have to ask themselves: “What risk level is acceptable for my business?” Often, this becomes a paralyzing exercise fraught with an unwillingness to change or, at best, test runs of cloud technology in safe areas like dev/test that are ultimately never implemented. This inaction leaves them behind more agile competitors who have clear, well-documented policies around adopting cloud technologies without adding significant risk. Lacking a defined cloud initiative does something that many companies may find surprising – it increases their risk and vulnerability as bad actors, security attacks, and attempts at gaining access to sensitive data become more sophisticated.
“What risk level is acceptable for my business?” This often becomes a paralyzing exercise fraught with unwillingness to change.
Well-architected cloud environments are the best solution to keep up with those security challenges. According to Gartner, “…through 2020, public cloud infrastructure as a service (IaaS) workloads will suffer at least 60 percent fewer security incidents than those in traditional data centers.” This additional security is the result of the major cloud platform providers (AWS, Azure, Google, and Alibaba) having a virtually unlimited budget and tight controls of the underlying platform. While they do provide a secure platform, it is still up to the users to architect secure environments. Gartner also states: “through 2022, at least 95 percent of cloud security failures will be the customer’s fault.”
The Way Forward
So, what can you do to ensure that your FDA-regulated business remains competitive and secure in a world where change is constant, and breaches happen daily? The first step is also one of the most important: Secure the understanding and sponsorship of the entire executive team.
There should be unanimous and clear support from the executive team, and a realistic understanding of the benefits of a cloud solution. Without their support, any adoption challenges may cause the project to stall, create doubt, or even lead to abandoning your cloud initiatives altogether.
Once you have the executive team’s support, a company-wide policy for cloud initiatives needs to be developed. This policy should be created by those with a deep knowledge of cloud computing to take full advantage of the appropriate cloud services for your business requirements. At this point, engaging with a managed service provider or consultant can be highly beneficial and ensure that your cloud initiatives are realistic and follow best practices for cost, security, and compliance requirements.
Developing Effective Adoption Policies
At minimum, a cloud adoption policy should address security and compliance requirements, workload elasticity and scaling demands, departmental ownership and responsibilities, risk assessment and remediation methodologies, and critical dependencies. In addition, you should also consider addressing storage retention, disaster recovery, or business continuity. The process of developing these comprehensive adoption policies allows your organization to gain a better understanding of how the cloud fits into each aspect of your business, while providing clear goals for your teams to pursue.
Having a clearly defined objective is best practice for implementing a cloud solution, but being too focused on the minutiae can lead to tunnel vision and increases the likelihood of creating an inflexible adoption plan. Designing a plan that functions more as a framework or a set of guidelines than a codified set of instructions, in a sense mirroring the flexible nature of the cloud, will help prevent your teams from losing sight of the advantages of cloud services or hindering innovation.
Another common pitfall to cloud adoption is the tendency to apply current, non-cloud policy to your cloud adoption initiatives. Adherence to legacy IT policies will prove challenging to cloud adoption and could make it impossible to fully realize the advantages of moving to a cloud solution. And outdated approaches could even result in greater costs, poor performance, and poorly secured environments. These risks can all be addressed with appropriate cloud-based policies that foster cloud-first approaches to new initiatives.
Becoming a secure, cloud-enabled organization requires consistent diligence from your internal teams and continuous adherence to the company cloud policy. In the end, the most significant risks to the security of your infrastructure are tied to your own policies and oversight, and the continued security of your cloud and data will require the involvement and cooperation of your entire organization. Clear communication and targeted training will help your teams understand their role in organizational security.
An Outsider’s Expertise
If you’re not sure about the effectiveness of your approach to cloud adoption, bringing in a third party to assist with policy creation or implementation can help save time and money while ensuring that best practice security is built into your approach. Outside organizations can also provide valuable assistance if you’ve already implemented cloud solutions, so it’s never too late to get guidance and insight from experts who can point out where processes or solutions can be improved, corrected, or optimized to meet your specific business requirements.
These third-party engagements have proven to be so useful that AWS has created the Well-Architected Framework and an associated Well-Architected Review program that gives their clients an incentive to have a certified third party review and then optimize their AWS solution (learn more about Effectual’s Well-Architected Review offering). Organizations such as the Society of Quality Assurance and the Computer Validation & Information technology Compliance (CVIC) (disclosure: I am a member of the CVIC) are also discussing these issues to provide guidance and best practices for Quality Assurance professionals.
Outside professional and managed services can provide an immense level of assistance through an objective assessment of your organization’s needs. Their focused expertise on all things cloud will lighten the load on your internal IT teams, help ease any fears you may have about cloud adoption, discover potential savings, and provide guidance to fortify the security of your cloud solution.
Mark Kallback is a Senior Account Executive at effectual, Inc.
Serverless Infrastructure Enables Data Access Related to Environmental Issues
Effectual worked with a Federal Government client to provide a mission critical solution that scientists could apply to classification tasks on large data sets related to earthquakes, volcanic activity, waterline erosion, and water quality.
This Federal Government customer looked to our team to migrate its on-premises infrastructure to a serverless infrastructure on AWS. The client was in need of a centralized data catalog, management solution for users, and data access for environmental issues.
We supported the client with a serverless solution that consisted of Amazon API Gateway, Amazon Cognito User Pools, AWS Lambda, and AWS Step Functions. This ensured the customer’s ability to make high-volume, complex data accessible to stakeholders, policymakers, and managers to facilitate data-driven conversations about environmental issues in a secure setting.
Our team implemented API Gateway to handle the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls to process any surge of traffic on its website.
Wey implemented AWS Cognito User Pools for control over user authentication and user access for the website. This allowed for secure token handling and management of authenticated users from all identity providers.
We implemented Lambda functions to run code in a serverless environment and process its large data sets related to environmental issues. The client was able to reduce cost by 80%.
TNTP’s mission is to end the injustice of educational inequality by providing excellent teachers to the students who need them most and by advancing policies and practices that ensure effective teaching in every classroom.
In the wake of a flooding, TNTP looked to Effectual to quickly and effectively move their public-facing web applications and internal applications to the AWS cloud for better cost, scalability, disaster recovery capabilities, and better application performance.
Effectual worked with TNTP to define a migration strategy, set up the infrastructure in accordance with best practices and to take advantage of the full feature set of cloud, and provided scripts to automate future updates and deployments. Effectual introduced TNTP to the Infrastructure as Code model so that they could version control the state of their infrastructure through the use of AWS CloudFormation templates and take advantage of AWS’ built-in resource dependency definitions to perform rolling updates with minimal downtime or system impact.
TNTP experienced lower costs for running their workloads in the cloud compared to on-premise IT hardware and maintenance costs. Effectual assisted TNTP to utilize cloud purchasing options and offerings to meet TNTP’s technical requirements while remaining cost-efficient.
The use of the AWS cloud provided capabilities for flexible infrastructure to allow accommodation of various sizes of workloads. The infrastructure used AWS Auto Scaling capabilities along with custom settings in AWS CloudWatch to automatically scale to accommodate larger workloads while retaining transparency of the scaling activities to the end user.
Failover capabilities and strategies such as the use of AWS Elastic Load Balancing within AWS were implemented to protect the system, maximize uptime, and minimizes data loss in the event of a disaster. Notifications, alarms, and safeguards were put in place to ensure immediate notification of any abnormal behavior.
Applications Rearchitected in AWS to Automate Security Triggers
Effectual led a Federal Government client in their journey from on-premises to AWS by extending their data center into the cloud and rearchitecting their applications.
Effectual provided guidance in the following areas
Implementing Automation for the client.
Creating a new AWS infrastructure and environment.
Updating and retooling current applications.
Building the solution as a receiver and retooling specific applications to function in the new environment.
Interpreting and providing additional information and understanding of features that are new and being developed as it pertains to their issues.
Our Team Leveraged the following technologies
AWS CloudFormation templates were created for DevOps
Organizations and AWS Config for management of the system
AWS CloudTrail and Amazon CloudWatch were utilized for automating security recommendations
Amazon CloudWatch was programmed to alert the client if changes were made in their system. The response would trigger the system to return to original configurations and alert security to these changes.
AWS infrastructure resources, EC2 instances and RDS database infrastructures
Migration to the Cloud
We rebuilt client applications in the AWS Cloud to connect to their on-premises data. This made their applications more accessible by all and created a working hybrid environment for their data.
We deployed AWS infrastructure services, including Amazon CloudWatch to monitor resources and trigger responses to changes in the environment.
Management of Resources
The services in AWS monitor both on-premises and AWS cloud environments. The time to build components in the environment was significantly reduced and instances were saved as a template for repeatability.
Considerations for AWS Control Tower Implementation
AWS Control Tower is a recently announced, console-based service that allows you to govern, secure, and maintain multiple AWS accounts based on best practices established AWS.
What resources do I need?
The first thing to understand about Control Tower is that all the resources you need will be allocated to you by AWS. We will need AWS Organizations established, an account factory to create accounts per LOB, and Single Sign On (SSO) to name a few. Based on the size of your entity or organization, those costs may vary. In the Control Tower precursor, AWS Landing Zones, we found that costs for this collection of service could range near $500-$700 monthly for large customers (50+ accounts), as deployed. Control Tower will probably be a similar cost, possibly higher depending on the size of your organization. I will address later in this post on how to go and use Control Tower once you have an account set up a Brownfield situation. In a perfect world, it would be nice to setup the Control Tower and in a Greenfield scenario, but sadly, 99% of the time, that’s not the case.
If you’re a part of an organization that has multiple accounts in different lines of business, this service is for you.
What choices do I need to make?
In order to establish a Cloud Enablement Team to manage Control Tower, you need to incorporate multiple stakeholders. In a large organization, that might entail different people for roles such as:
AWS Solution Architect
Cloud Engineer (Automation)
You want to be as inclusive as possible in order to get the most breadth of knowledge. These are the people that will be making the decisions you need to migrate to the cloud and then most importantly, thrive once present and remain engaged. We have the team, so now what can we do to make Control Tower work the best for us?
Decisions for the Team
1. Develop a RACI
This is one of the most crucial aspects of Operations. If you do not have accountability or responsibility, then you don’t have management. Everyone must be able to delineate their tasks from the rest of the team. Finalizing everyone’s role in the workflow then this will solve a lot of issues before they happen.
2. Shared Services
In the shared services model, we need to understand what resources are going to the cloud and what will stay. Anything from Active Directory to DNS to one-off internal applications will have to be figured out in a way to accommodate functionality and keep the charge back model healthy. One of Control Tower’s most redeeming and worthy qualities is knowing what each LOB is costing and how they are helping the organization overall.
3. Charge Backs
Since the account factory (previously called Account Vending Machine) is established, each LOB will have its own account. In order to see what the LOB costs are, you must have an account. AWS does not do pricing based on VPC, but by account. Leveraging Control Tower, tagging, and third-party cost management resources all can combine to give an accurate depiction of the costs incurred by a specific line of business.
Security will have all logs dumped from each account into a centralized log bucket can be pointed to the tool of choice to analyze those logs. Other parties may perform audits to read your logs using ready only functions in an account that has nothing else, another feature of Control Tower. The multi-account strategy not only allows for better governance, but also now helps in case of compromise. If one account has been compromised, then the blast radius for all the other accounts is minimal. Person X may have accessed a bucket in a specific account, but they did not access it anywhere else. The most important thing to remember is that you cannot treat cloud security like data center security.
There are plenty of choices to make as it relates to Control Tower moving forward for an organization, but if you plan correctly and make wise decisions, then you can secure your environment and keep your billing department happy. Hopefully this has helped you see what it takes in the real world to prepare. Good luck out there!
Network Virtualization – The Missing Piece of Digital Transformation
The cloud revolution continues to impact IT, changing the way digital content is accessed and delivered. It should come as no surprise that this revolution has affected the way we approach modern networking.
When it comes down to it, the goal of digital transformation is the same for all organizations, regardless of industry: increase the speed at which you’re able to respond to market changes and evolving business requirements, improve your ability to adopt and adapt to new technology, and enhance overall security. Digital strategies are maturing, becoming more thoughtful and effective in the process, as organizations understand that the true value of cloud adoption and increased virtualization isn’t just about cost savings.
Technology is more fluid than ever, and dedicated hardware is limiting individual progress and development more and more every day. Luckily, cloud and virtualized infrastructure have helped lay the groundwork for change, giving companies the opportunity to more readily follow the flow of technological progress but, in the same way that a chain is only as strong as its weakest link, these same companies are only as agile their most rigid component. And that rigid chokepoint, more often than not, is hardware-based network infrastructure.
A Bit of History We likely wouldn’t have the internet like we do today if not for the Department of Defense needing a way to connect large, costly research computers across long distances to enable the sharing of information and software. Early computers had no way to connect and transmit data to each other. The birth of ARPANET in 1969, the world’s first packet-based network, and it’s ensuing expansion was monumental in creating the foundation for the Information Age.
The Case for Virtualization
While some arguments can still be made about whether a business might benefit more from traditional, hardware-based solutions or cloud-based options, there’s an inarguable fact right in front of us: software moves faster than hardware. This is what drove industries toward server and storage virtualization. However, network infrastructure still tends to be relegated to hardware, with the same manual provisioning and configuration processes that have been around for decades. The challenge of legacy, hardware-based network infrastructure is a clear obstacle that limits an organization’s ability to keep up with changing technologies and business requirements.
The negative effect of hardware-based networking goes beyond the limitation of speed and agility. Along with lengthy lead times, the process of scaling, modifying, or refreshing network infrastructure can require a significant amount of CapEx since you have to procure the hardware, and a significant amount of OpEx since you have to manually configure the newly acquired network devices. In addition, manual configuration is well-known to be error-prone, which can lead to connectivity issues (further increasing deployment lead time) and security compromises.
Networking at the Speed of Business and Innovation
As organizations move away from silos in favor of streamlined and automated orchestration, approaches to network implementation need to be refreshed. Typical data center network requests can take days, even weeks to fulfill since the hardware needs to be procured, configured (with engineers sometimes forced to individually and painstakingly configure each device), and then deployed.
Software-defined networking (SDN), however, changes all of that. With properly designed automation, right-sized virtual network devices can be programmatically created, provisioned, and configured within seconds. And due to the reduced (or even fully eliminated) need for manual intervention, it’s easier to ensure that that newly deployed devices are consistently and securely configured to meet business and compliance requirements.
Automation allows networking to match the pace of business by relying on standardized, pre‑defined templates to provide fast and consistent networking and security configurations. This lessens the strain and burden on your network engineers.
Modern IT is focused on applications, and the terminology and methods for implementing network appliances reflect that – but those applications are no longer tied to the physical data center. Sticking to a hardware-focused networking approach severely restricts the mobility of applications, which is a limitation that can kill innovation and progress.
Applications are not confined to a single, defined location and maturing digital and cloud strategies have led to organizations adopting multiple public and private clouds to achieve their business requirements. This had led to an increase in applications being designed to be “multi-cloud ready.” Creating an agile network infrastructure that extends beyond the on-premises locations, matching the mobility of those applications, is especially critical.
Network capabilities have to be able to bridge the gap from functioning consistently across all locations, whether they’re hardware-based legacy platforms, virtual private cloud environments, or pure public cloud environments.
This level of agility is beneficial for all organizations, even if they’re still heavily invested in hardware and data center space, because it allows them to begin exploring, adopting, and benefiting from public cloud use. Certain technologies, like VMware Cloud on AWS, already enable organization to bridge that gap and begin reaping the benefits of Amazon’s public cloud, AWS.
According to the RightScale 2019 State of the Cloud Report from Flexera, 84% of enterprise organizations have adopted a multi-cloud strategy, and 58% have adopted a hybrid cloud strategy, utilizing both public and private clouds. On average, the respondents reported using nearly five clouds on average.
A Modern Approach to Security
Digital transformation creates fertile ground for new opportunities – both business opportunities and opportunities for bad actors. Since traditional approaches to cybersecurity weren’t designed for the cloud, cloud adoption and virtualization have contributed to a growing need to overhaul information security practices.
Traditional, classical network security models focused on the perimeter – traffic entering or leaving the data center – but, as a result of virtualization, the “perimeter” doesn’t exist anymore. Applications and data are distributed, so network security approaches have to focus on the applications themselves. With network virtualization, security services are elevated to the virtual layer, allowing security policies to “follow” applications, maintaining a consistent security configuration to protect the elastic attack surface.
But whether your network remains rooted in hardware or becomes virtualized, the core of your security should still be based on this: Security must be an integral part of your business requirements and infrastructure. It simply cannot be bolted on anymore.
Picking the Right Tools and Technology for the Job
Choosing the right tools and technology to facilitate hybrid deployments and enable multi‑platform solutions can help bridge the gap between legacy systems and 21st century IT. This level of interoperability and agility help make cloud adoption just a little less challenging.
Addressing the networking challenges discussed in this post, VMware Cloud on AWS has an impressive set of tools that enable and simplify connectivity between traditionally hosted on-premises environments and the public cloud. This interconnectivity makes VMware Cloud on AWS an optimal choice for a number of different deployment use cases, including data center evacuations, extending on-premises environments to the public cloud, and improving disaster recovery capabilities.
Developed in partnership with Amazon, VMware Cloud on AWS allows customers to run VMware workloads in the cloud, and their Hybrid Cloud Extension (HCX) enables large-scale, bi-directional connections between on-premises environments and the VMware Cloud on AWS environment. In addition, VMware’s Site Recovery Manager provides simplified one-click disaster recovery operations with policy-based replication, ensuring operational consistency.
If you’re interested in learning more about VMware Cloud on AWS or how we can help you use the platform to meet your business goals, check out our migration and security services for VMware Cloud on AWS.
Ryan Boyce is the Director of Network Engineering at effectual, Inc.
Interview with an AWS Champion Authorized Instructor
James Hirmas is co-founder of JHC Technology, an Effectual Company, and an Information Technology Subject Matter Expert with nearly 20 years of experience.
James was named an Amazon Web Services (AWS) Champion Authorized Instructor in April 2019 by demonstrating his deep understanding of AWS services and solutions and his ability to teach AWS content to others. We had the chance to discuss with James what it means to him to be an AWS Champion Instructor.
What certifications do you have?
I have earned the following AWS certifications in addition to becoming an AWS Champion Authorized Instructor:
AWS SysOps Administrator
AWS Solutions Architect
AWS Professional DevOps Engineer
AWS Security Specialty
Why did you want to become an AWS Champion Instructor?
I pursued becoming AWS Champion Authorized Instructor to help mentor and set an example for the JHC Technology team. As one of the co-founders of JHC Technology, I strongly believe that we should always lead by example. One of the foundational principles JHC Technology is built upon, as defined in our JHC ethos, is “Never stop Learning.” At JHC Technology, we truly embrace this principle by providing continuous training to our teams and deeply entrenching everyone in cloud technology.
I also wanted to give back to the AWS community. Over the past 10 years working on the AWS platform, cloud technology has given me a tremendous opportunity to help build a successful business, reshape/redefine the technology landscape, and advance my professional career. As an AWS Champion Authorized Instructor, I work with a very diverse group of organizations from different industries and I can impart my decade-plus of real-world AWS experience to further promote, educate, and accelerate organization’s journey to AWS.
What was the process to become an AWS Champion Instructor?
In order to become an AWS Authorized Champion Instructor, I recommend multiple years of experience deploying advanced workloads on AWS. According to the AAI program guide, you need to hold at least four valid AWS Certifications at the Associate or above level, with at least one of them being a Professional level certification. You also need to work with an AWS partner organization that is in the AWS Training Program and you will need to complete an Instructor Delivery Workshop. The workshop demonstrates technical knowledge and ability to deliver an AWS authorized course and/or complete a co-teach overseen by a qualified AWS Mentor Instructor. AWS has multiple official courses that are offered from Architecting on AWS to Security Operations on AWS. For each course offered, the AWS Authorized Champion Instructor needs to have the appropriate mix of AWS certifications.
Describe your experience as an AWS Authorized Instructor.
So far, my experience has been very rewarding. I have been able to travel to different parts of the United States and other countries to provide AWS training. I have been able to provide value with real work use cases and experiences.
It is an investment when you become an AWS Authorized Champion Instructor. I spend a lot time when I am not training reviewing course material, preparing examples to present in class, and constantly learning new AWS features and use cases.
FISMA Compliance Requirements Met for Self-Service Cloud Solution
Effectual enabled a Federal Government customer to set up a self-service cloud solution which is secure, compliant, and automated to scale up and down as necessary.
The Customer wanted to scale out compliant accounts to meet security concerns such as accessing only approved services, protecting centrally managed resources, and ensuring logging and change activity was being captured. The overall issue was ability to consistently provision AWS accounts in a scalable fashion and manage them over time, keeping them up-to-date with newly approved AWS Services. The goal was to provide secure and compliant cloud hosting options while setting up a customer self-service solution.
We assisted the client in creating their entire environment from Infrastructure as Code while implementing a strict change control processes via GitLab. Custom pipelines were created based off the CI/CD framework for structured code. Overall the entire process was automated, eliminating the scalability issue of provisioning accounts. Our resources worked directly alongside the agency resources to document and achieve a FISMA Moderate ATO.
The customer was able to quickly provision accounts in a consistent method across multiple geographical locations and regions. The entire environment can be deployed in one hour.
We enabled the customer to securely provision their own infrastructure, standardized methodology, and least-privileged architecture. This methodology ensures security in the cloud for the client.
Management of Resources
The services in AWS monitor both on-premises and AWS cloud environments. The time to provision new accounts was reduced from a month to one minute. The deployments are now consistent and can be saved for later use.
Effectual delivered a mission-critical solution to a client that ensured the delivery of UAV imagery taken from infrastructure towers that were used to alert high risk fire areas of a wildfire and other natural disasters.
Our customer required a move from its on-premises infrastructure to a centralized Cloud environment in AWS. They looked to us to handle high availability architecture and fault tolerance to meet workloads over many geographical locations. We automated common activities such as change requests, monitoring, patch management, security, and backup services, and provided full-lifecycle services to provision, run, and support enterprise infrastructure.
We provided a client with Technical Amazon Web Services Infrastructure architecture to deliver a comprehensive, secure, and cost-effective hosting solution for supporting their efforts with Pacific Power. In addition, our team delivered Managed Services for the customer’s AWS environment. This assisted with the client’s ability to deploy drones to inspect the infrastructure of electrical towers and ensure their efficiency in wake of natural disasters.
The implementation of Amazon CloudWatch Events for serverless workflow to trigger Lambda functions. Drones are programmed to deploy and inspect electrical towers to ensure that they are performing correctly.
We created a proprietary AWS-hosted solution in order for the customer to lower costs by running their workloads in the cloud. Our team assisted the client to utilize cloud purchasing options and offerings to meet their technical requirements while remaining cost-efficient.
We configured EC2 instances that ensure adequate capacity to meet traffic demands and compute capacity. The implementation of automated launch configurations to allow the client to quickly launch and/or scale application severs in target environments in the future.
One year ago, our team made an investment into a self-hosted installation of GitLab.
We had been successful in delivering a managed GitLab installation at a customer site and saw the value in taking advantage of everything the platform had to offer for our internal workloads. As an AWS DevOps Competency partner, we have a successful track record of helping organizations adopt DevOps processes and we understand that the biggest challenge is often aligning an organization’s culture with DevOps principles.
GitLab’s biggest strength is that it addresses all stages of the software development lifecycle. GitLab’s features align strongly with the stages and principles our team has outlined in our DevOps process. The cornerstone of this DevOps process is that everything is delivered as code and all code is continuously version controlled, tested, and cross-checked by peers. The marriage of GitLab’s repository tools with their built-in CI platform eliminates much of the overhead of setting up continuous integration and testing. Our team has built custom pipeline templates specifically designed around deployments using AWS, CloudFormation, Docker, Kubernetes, Terraform, and other platforms. These pipeline templates allow new projects to inherit shared knowledge and hit-the-ground-running to deliver operational excellence with Agile development speed. We’ve also committed ourselves to sharing these templates and learned best practices with the community to aid others in quickly and efficiently adopting GitLab and cloud and driving new development.
Our team has designed a one-click style deployment of GitLab on AWS with high availability and security out-of-the-box. We’re using this solution to help other organizations rapidly adopt GitLab and have been successful in doing so at several government and commercial organizations. We also have a one-click GitLab Runner on AWS solution available for scalable, secure GitLab CI runners and are actively working on a one-click deployment for GitLab Runner on Azure and GCP.
GitLab has been a cornerstone of our DevOps practice, and we are just getting started. We have empowered organizations to automate software testing and deployments using GitLab as the engine, and organizations have been able to move faster and better address end-users with those abilities. We’re excited to see what organizations can do with the power that DevOps’ operational excellence gives them, and we’ve partnered with GitLab to accelerate them along that journey.
If you or your organization has more questions in regards to GitLab or our DevOps process, reach out to [email protected] to set up some time to chat about your business goals.
During the past nine years of delivering cloud solutions to government and industry, our team has identified a gap in the delivery of Managed Service Provider (MSP) solutions in a cloud environment.
As an AWS Premier Consulting Partner with the DevOps, Government, and Non-Profit competencies, as well as the GovCloud Skill Partner Status, there are few workloads that we haven’t helped customers migrate and manage in a cloud environment. We also deliver these solutions through Microsoft Azure, where we are a Silver Cloud Partner.
Our focus with MSP is around a structured, repeatable, five-step MSP process
We previously discussed the Evaluation, Automation, Optimization, and Monitoring phases as part of the MSP process. In this blog – the final in a series of five on Next Generation MSP (NG-MSP) – we’ll touch on Integration, a critical component of a successful MSP strategy as it brings everything together. As providers of cloud based MSP, we focus our efforts in automation, repeatability and collaboration, as those same aspects in an on premise MSP environment is where a lot of capability is lost.
With the flexible nature of cloud vs. on-premise, the automation enables an NG-MSP provider like Effectual to react quicker and with more structure because resources can be located anywhere and the infrastructure backend is handled by AWS. You begin to gain efficiencies as a result of technology giving the ability to move quicker, simpler, and cheaper than an on-premise environment. The traditional on-premise environment will require physical adjustments to infrastructure, may require additional infrastructure to be ordered, and is dependent on specific workers rather than a pool of resources coupled with packaged, scripted activities.
At the integration stage, we are implementing the Effectual toolchain with a focus on meeting Continuous Integration/Continuous Deployment (CI/CD) objectives. These tool sets include GitLab and JIRA integration, among other components. With the cloud-based tool chain in place, we can offload 70%-80% of current workloads as it relates to an on-premise environment. However, as an NG-MSP provider, we’re not changing the workflow but simplifying the process of getting to production.
We’ve done this long enough to know that NG-MSP models cannot be a one-size fits all proposition. In our cloud-model, as we get to the Integration component, we are leveraging best-in-breed toolchains to ensure security, efficiency, and an optimized infrastructure to support our customer workloads.
We stand ready to support your organization’s cloud environment with our NG-MSP services. To discuss options, please reach out to [email protected].
Across every industry, cloud-native businesses are disrupting legacy institutions that have yet to transform traditional IT platforms.
To remain competitive, industry giants need to change the way they think about technology and adopt a cloud-first mindset. Prioritizing cloud-based solutions and viewing them as the natural, default option is vital to the success of new projects and initiatives.
Migrating legacy systems to cloud has the added benefit of eliminating technical debt from older policies and processes. However, it is important to be mindful in order to avoid creating new technical debt when developing and deploying cloud systems. While adopting a cloud-first mindset may seem like an expected result of digital transformation, it requires significant changes to an organization’s culture and behavior, similar to those required for the effective adoption and implementation of DevOps methodologies.
We have to rethink the old way of doing things – cloud is the new normal
Evolving needs and capabilities
When “cloud” first entered the lexicon of modern business, it was incorrectly thought of as a cost‑cutting measure. Organizations were eager to adopt the cloud with the promise of savings – despite not fully understanding what it was or its ever-growing capabilities. These types of implementations were generally short-sighted: lacking a well-defined digital strategy and focused on immediate needs rather than long-term goals.
As adoption increased, it became apparent that adjusting approach and redefining digital strategy is necessary for success. Optimizing applications for the cloud and developing comprehensive governance policies to rein in cloud sprawl, shadow IT, and uncontrolled (and unmonitored) spend are just part of the equation.
A cloud-first approach reshapes the way an organization thinks about technology and helps mitigate the potential to recreate unnecessary technical debt that was eliminated through digital transformation initiatives.
The human element of digital transformation
Digital transformation should extend beyond technology. It’s a long-term endeavor to modernize your business, empower your people, and foster collaboration across teams. Transforming your systems and processes will have a limited impact if you don’t also consider the way your teams think, interact, and behave. This is especially important because the significant operational changes introduced by modernizing infrastructure and applications can present challenges to employees who feel comfortable maintaining the status quo. Before you can disrupt your industry, you have to be willing to disrupt the status quo within your own organization.
The fact is that change can be difficult for a lot of people, but you can ease the transition and defuse tension by actively engaging your teams. You cannot overstate the importance of clear, two-way communication. Letting your people know what you’re planning to do and also why you’re doing it can help them understand the value of such a potentially massive undertaking. It’s also important to have a solid understanding of what your teams need and creating open lines of communication will enhance requirements gathering efforts. This level of communication ensures that whatever you implement will adequately address their needs, and ultimately improve their workflow and productivity.
The introduction of new tools and technologies, even if they’re updated versions of the ones currently in use, will generally require some level of upskilling. Helping your teams bridge the technical gap is a necessary step.
Competition at its finest
Few sectors have seen the level of disruption faced by the finance industry. FinTech disruptors, born in the cloud and free from the chains of technical debt and bureaucratic overhead, have been able to carve out their place in the market. They’ve attracted customers by creating innovative offerings and customer-focused business models, competing with legacy institutions that seemed to have an unassailable dominance that barred any new entrants.
Legacy retail banking institutions, known for being risk averse, had a tendency to implement new technology very slowly. They were plagued by long development cycles, dedicated hardware solutions, and strict compliance requirements to safeguard highly sensitive data.
When Capital One turned its attention to the cloud, they created a holistic digital strategy that wasn’t limited to tools and systems. They understood that technology was not a line item on a budget, but an investment in the company’s future, and that successfully executing their strategy would require a culture shift. They focused on attracting technologists who could enhance the company’s digital capabilities to increase employee engagement, improve cybersecurity, and improve customer experience by using the latest technologies, including artificial intelligence and machine learning. They also created a cloud training program so their employees would understand the technology, regardless of whether or not they were in technical roles, reinforcing the company’s cloud-first mindset.
FinTech disruptors, born in the cloud and free from the chains of technical debt and bureaucratic overhead, have been able to carve out their place in the market.
Understanding your options
Developing a proper cloud-first mindset is not about limiting your options by using the cloud exclusively. A digitally transformed business doesn’t adopt the latest technology simply for the sake of adoption. In fact, the latest and greatest SaaS or cloud-based offerings may not always be the best option, but you have to know how to make that determination based on the unique needs and circumstances of your business. By objectively assessing business goals, considering all options (including traditionally hosted), and prioritizing agile, flexible solutions, you can redefine your approach to problem-solving and decision-making. This mindset means that cloud is no longer the “alternative.”
We have to rethink the old way of doing things – cloud is the new normal, and hardware-based options should only be implemented if they are truly the best way to meet business goals and overcome challenges. We don’t need to abandon on-premises or traditional IT to maintain or regain competitive edge. We just need to understand that it’s not always the right choice.
This approach will help you develop a macro view of your organization’s needs and prompt you to identify and treat the underlying cause of business challenges, not just the symptoms.
Building a foundation for disruption
Becoming a disruptor in your industry is not the goal of digital transformation –that takes more than just adopting the cloud. The goal is to free your organization from the restraints of costly, outdated legacy infrastructure and monolithic applications, and enabling your teams to scale and innovate. The flexibility of cloud and SaaS-based options reduce the risks associated with developing new products and services for your customers, and instilling a culture of cloud-first thinking gives your people the freedom to explore and experiment. That’s how you drive innovation and compete against new, lean born-in-the-cloud competitors. That’s how you disrupt.
RFD & Associates, Inc., is an IT Technical Services Company with over 30 years of experience delivering IT solutions to public and private sector clients.
RFD delivers solutions from Mainframe to Mobile and everything inbetween. They have helped hundreds of organizations design, build, purchase and implement optimal technology solutions to achieve business goals. RFD needed help designing and developing a scalable, Amazon Web Services (AWS) cloud hosted, multi-tenant web and mobile friendly application. The proposed solution had a requirement to integrate with external APIs to ensure flexibility for future enhancements and integration with third-party tools. The application was also required to be compliant with Personally Identifiable Information (PII) and the U.S. Health Insurance Portability and Accountability Act (HIPAA) security.
Effectual Provided Guidance in the following areas
AWS design and architectural services to include making RFD’s multi-tenant hosting environment PII/HIPAA compliant
Provided AWS Training and best practices guidance on how to leverage AWS resources
Assisted in helping RFD achieve its defined goals:
Identify the challenges presented in third-party hosting of AWS.
Evaluate the use of cloud services to meet RFD business and technical requirements.
Determine portable containerization services.
Evaluate architectural decisions in AWS Commercial and GovCloud Regions.
A four-phased approach was developed to implement an AWS hosted environment for RFD:
Phase 1: Discovery, AWS Service Selection, and PII/HIPAA Security Requirements Determination.
Phase 2: AWS Foundation Build. Provisioned appropriate environments and access; established AWS accounts
Phase 3: AWS Service Build. Provisioned AWS services to include: EC2, Route53, S3, WAF, etc.
Phase 4: Process Documentation and Environment Review. Created AWS documentation of resources and provided reports on overall solution, security and cost.
We configured EC2 instances that are PII/HIPAA compliant ensuring adequate capacity to meet traffic demands and compute capacity. In addition, we implemented automated launch configurations to allow RFD to quickly launch and/or scale application severs in target environments in the future.
Security & Compliance
The implementation of AWS Compute, Storage, and PII and HIPAA compliant Database services to ensure the security of sensitive data used in the environment.
To maximize the functionality of many services, AWS CloudWatch was configured to help RFD set thresholds/alarms to monitor custom metrics for auto-scaling needs.
In the borderless and elastic world of the cloud, achieving your security and compliance objectives requires a modern, agile, and autonomous security strategy that fosters a culture of security ownership across the entire organization.
Traditional on-premises approaches to cybersecurity can put organizations at risk when applied to the cloud. An updated, modern strategy for cloud security should mitigate risk and help achieve your business objectives.
Cloud service providers such as Amazon Web Services (AWS) place a high priority on the security of their infrastructure and services, and are subject to regular, stringent third-party compliance audits. These CSPs provide a secure foundation, but clients are still responsible for securing their data in the cloud and complying with their data protection requirements. This theory is substantiated by Gartner, which estimates that, through 2020, workloads hosted on public cloud will have at least 60% fewer security incidents than workloads hosted in traditional data centers, and 95% of cloud security failures through 2022 will be the fault of customers.
Traditional approaches to cybersecurity weren’t designed for the cloud – it’s time for an update
Updating how you think about cybersecurity and the cloud
Despite the significant security advances made by CSPs since the birth of cloud, users still need a deep understanding of the cloud’s shared responsibilities, services, and technologies to align information security management systems (ISMS) to the cloud. Today, the majority of data breaches in the cloud are the result of customers not fully understanding their data protection responsibilities and adopting poor cloud security practices. As the public cloud services market and enterprise-scale cloud adoption continues to grow, organizations must have a comprehensive understanding of not just cloud, but cloud security specifically.
Through 2022, at least 95% of cloud security failures will be the customer’s fault
A poor grasp of the core differences between on-premises and cloud technology solutions resulted in a number of misconceptions during the early days of cloud adoption. This lack of understanding helped fuel one of the most notable and pervasive cloud myths in the past: that it lacked adequate security. By now, most people have come to realize that cloud adoption and digital transformation do not require a security tradeoff. In fact, the cloud can provide significant governance, risk, and compliance (GRC) advantages over traditional on-premises environments. A cloud-enabled business can leverage the secure foundation of the cloud to increase security posture, reduce regulatory compliance scope, and mitigate organizational responsibilities and risk.
It is common to see enterprise organizations lacking the necessary expertise to become cloud resilient. Companies can address this skills gap through prescriptive reference architectures. AWS, for example, has created compliance programs for dozens of regulatory standards, including ISO 27001, PCI DSS, SOC 1/2/3, and government regulations like FedRAMP, FISMA, and HIPAA in the United States and several European and APAC standards. Beyond these frameworks, consultants and managed service providers can work with organizations to provide guidance or architect environments to meet their compliance needs.
Regardless of the services leveraged, the cloud’s shared responsibility model ensures that the customer will always be responsible for protecting their data in the cloud.
Making the change
Similar to the challenges and benefits of implementing DevOps (discussed here by our CEO, Robb Allen), effective cloud security requires a culture change with the adoption of DevSecOps, shifting the thinking in how teams work together to support the business. By eliminating the barriers between development, operations, and security teams, organizations can foster a unified approach to security. When everyone plays a role in maintaining security, you foster collaboration and a shared goal of securely and reliably meeting objectives at the speed of the modern business.
Additionally, cloud-specific services and technologies can provide autonomous governance of your ISMS in the cloud. They can become a security blanket capable of automatically mitigating potential security problems, discovering issues faster, and addressing threats more quickly. These types of services can be crucial to the success of security programs, especially for large, dynamic enterprises or organizations in heavily regulated industries.
Implementing the right cloud tools can lead to significant reductions in security incidents and failures, giving your teams greater freedom and autonomy to explore how they use the cloud.
The way to the promised land
Security teams and organizations as a whole need to have a deep understanding of cloud security best practices, services, and responsibilities to create a strategic security plan governed by policies that align with business requirements and risk appetite. Ultimately, however, a proper cloud security strategy needs buy-in and support from key decision makers and it needs to be governed through strategic planning and sound organizational policies. Your cloud security strategy should enable your business to scale and innovate quickly while effectively managing enterprise risk.
Darren Cook is the Chief Security Officer of effectual, Inc.
“Celebrate achievement, get better every day – this is FinOps.”
—Mat Ellis, CEO of Cloudability
Introducing the day, Mat set the stage that public cloud adoption is part of a much bigger trend seen in many industries throughout history – managing a supply chain. Milestone innovations disrupt at an astronomical scale; from the printing press, to rubber, to the internet, and now cloud computing. We’ve all felt the disruption created by cloud computing and many of us have been part of the 21st Century IT revolution. As seen at AWS re:Invent last year, the adoption of DevOps culture to foster innovation and enable competitive advantage has been embraced by large insurance organizations like Guardian and the world famous guitar manufacturer Fender.
However, with AWS now 13 years old, many cloud technology buying decisions are still based on an outdated model. There is a need for iterative, ongoing monitoring and accounting for cloud spend. Enter Cloudability. Analyzing hundreds of millions in cloud spend per month, and billions per year, Cloudability’s platform delivers keen insights and benchmarking tools that enable a clear path to cloud cost diligence and FinOps success.
Cost Management in The Cloud Age
Digging into the data behind the mission of FinOps, Owen Rogers, Research Director, 451 Research presented some stark realities about the current state of cloud cost management (Full report available here). The study found that more than half of large enterprises worry about cloud costs on a daily basis and 80% believe that poor cloud financial management has a negative impact on their business. These enterprises need a comprehensive platform to manage multi-million-dollar cloud budgets.
Owen Rogers, Research Director, 451 Research presented some stark realities about the current state of cloud cost management
Another eye-opening data point presented was that 85% of respondents overspend their budgets, with nearly 10% spending two to four times their allocated budget. Pair this with 18% of respondents that were unaware they were overspending, and the picture is not pretty. The biggest reasons cited for not addressing this issue were “too small of an overspend to resolve” and “not wanting to hinder innovation.”
While well-intentioned, the study showed that “not wanting to hinder innovation” and pushing off a responsible approach to cloud cost management does exactly what the respondents are trying to avoid: halts cloud adoption, cripples innovation, lowers quality of service, increases cost, and creates a sprawling underutilized cloud footprint.
Cloud Cost Management Directly Impacts Company Culture and Business Bottom Line
The reality is that cloud cost management directly impacts business. Thankfully, there are steps to take to mitigate the commonplace inefficiencies identified by Owen. For example, 33% of respondents are manually extracting and aggregating cloud costs in a spreadsheet – this is the epitome of anti-agile. Only 52% of instances are rightsized for their workload and, beyond that, only 52% of respondents are taking advantage of Reserved Instance discounts.
The tools and opportunities to improve the health and efficiency of your cloud environments are readily available. In fact, the 451 Research report shows average savings of 27% were achieved through the use of a cost management platform. With an expected CAGR of 17% between 2017-2022, now is the time to implement the behavioral changes that instill a culture of FinOps within your organization.
The problem is shared accountability – The solution is a FinOps culture
What became apparent in the research presented by Owen Rogers is a distinct need for IT and Finance teams to come to the table together to discuss the path forward. The good news is there are companies that are pushing the envelope and leading the way in diligent and responsible cloud cost management. Those who have embraced a FinOps culture are utilizing performance benchmarking and have a clear understanding of the fully-loaded costs of their cloud infrastructure. This is the promise that we can aspire to and it starts with collaboration between IT, finance, and individual lines of business.
There is a distinct need for IT and Finance teams to come together to discuss the path forward
FinOps high performers have near real-time visibility of all cloud spend. Individual teams understand their portion of total spend, are enabled to budget and track against targets, and utilize Reserved Instances for 80-95% of their cloud services.
Similar to having a clear understanding of household finances, this level of diligence affords more benefits than just cost savings. A remarkable side effect of FinOps culture is a 10-40% improvement in operational efficiency within your organization.
In addition to the information presented at the Cloud Economic Summit, Cloudability launched the FinOps Foundation. Comprised of founding members from Atlassian, Nationwide, Spotify, Autodesk, letgo, and many others, the FinOps Foundation is a non-profit trade organization bringing people together to create best practices around cloud spend.
J.R. Storment, Cloudability Co-founder, takes on the role of President of The FinOps Foundation. J.R. describes the need for the organization here.
We are excited to see our partner defining this space and eager to participate in the FinOps Foundation. We are also looking forward to reading “Cloud Financial Management Strategies, Creating a Culture of FinOps,” their O’Reilly Media book which is slated to be published later this year.
Thanks again to Cloudability for hosting us at the event, we are looking forward to an exciting year together.
The cloud partner ecosystem is changing. The days when organizations could act as a “Jack of all trades, master of none” are over.
Legacy IT resellers are going the way of the dinosaur in favor of partners who can deliver clear value-add with a track record of transformative success. Specialization is the order of the day. This cuts to the heart of what we mean by a partnership — and how it differs from simply being a “vendor.”
IT partnerships should allow your in-house team to remain focused on generating revenue and building the organization.
Why Specialized Partnerships Matter
Choosing the right partner is absolutely critical to executing a successful cloud transformation. We addressed this in a previous post. Every organization is necessarily limited by its own technical and human resources. The right partner brings expertise, experience, and proven processes to ensure that internal limitations don’t equal a failed transformation process.
A Successful Cloud Partnership
Let’s take a look at one of the most recent and important cloud partnerships: AWS and VMware. AWS brought their cloud platform services together with VMware’s virtualization expertise. The result was a specialized certification program, a robust migration service, a cost insight tool providing greater transparency into cloud spending, and a joint hybrid cloud product to incentivize customer adoption. Each partner brought highly specific value-add services and together they created a game-changing cloud solution for the enterprise.
Partners Versus Vendors
It’s worth exploring what we mean when we talk about being a partner as opposed to being a vendor. A vendor is easy enough to explain: It is a company providing a service. The point is, even the best vendors are not as invested in your success as a partner. They certainly wish their customer success and hope for continued business, but there is no strategic, long-term involvement, or commitment to understanding their clients’ unique business goals.
In some cases, vendors may even push templated or cookie-cutter solutions that simply don’t fit. This isn’t to say that every vendor is out to take advantage of their customers; it’s simply a recognition that a generalized vendor offering tends to be limited, in contrast to a specialized partnership.
By comparison, a successful partnership is a more intimate relationship. In these engagements you’re not just purchasing IT services – you’re working hand-in-hand to grow the efficiency and effectiveness of your IT resources.
Cloud security, migration, and cost optimization are exactly the types of endeavors that call for partners.
The key difference is a subtle but important one — collaboration. It’s often thought that a good partner will “take care of everything” for you, but this is not true, nor should it be. A true partner requires your input to understand how your business defines success, and relies on this data to make informed decisions on the technologies they deploy. It is essential for your teams to be involved in this process, as they will adopt and learn new methodologies and processes throughout the engagement.
It’s not about choosing between vendors or partners. It’s about recognizing where more generalized vendors will fulfill your needs and where specialized partners are a better fit. Simple, straightforward tasks are fine for vendors. More involved and strategic endeavors, however, require a partner. Cloud security, migration, and cost optimization are exactly the types of endeavors that call for partners.
Extending Your In-House Capabilities
IT partnerships should allow your in-house team to remain focused on generating revenue and building the organization. Leaning on strong partners can in effect be an extension of your IT team, expanding your resources and solving problems that may have required additional training or experience beyond the expertise and skill sets of your internal teams.
Cloud security, migration, and cost optimization are exactly the types of endeavors that call for partners.
Keeping your teams focused on their core responsibilities has a highly desirable secondary effect – boosting in-house morale. Not only does this improve the workplace, it makes it easier for you to attract and retain top talent.
At effectual, we engage you as a partner, not a vendor, which is why we specialize in cloud, and not cloud-adjacent services like data center operations. Our deep experience in cloud enablement facilitates your digital transformation. This includes helping you to determine the best implementation strategy as well as establishing metrics to quantify and measure your success. But our specialization is in security and financial optimization.
Cloud is a rapidly evolving ecosystem. AWS rolled out 1,400 new services in 2017, another 800 throughout the first half of 2018, and an impressive number of new product and service announcements during re:Invent 2018. We understand that it can be difficult to wade through these waters to find the right solutions for your business challenges, including your specific security requirements. What’s more, your team is likely already fully committed running core applications and tools. You need a partner who can help keep your in-house team free to do what it is they do best.
RightScale’s 2018 State of the Cloud reportfound that most organizations believed they were wasting 30 percent of their cloud spend. In fact, the study found that 35 percent of their cloud spend was attributed to waste. We look for ways to help our partners get their invoices under control but also to understand what is driving their cloud costs. Finally, we help organizations properly allocate their spend, ensuring that the right applications, business units, regions, or any other grouping of your business is spending exactly what it should and no more.
We strive to understand your long- and short-term goals by working closely with your organization and provide you with strategic solutions for sustained growth. Interested in learning more? Reach out and let us know what you are looking to solve – we love the hard questions.
If you have been to AWS’s re:Invent, then you know the tremendous amount of excitement that cloud evangelists experience during that time of the year.
The events that AWS hosts in Las Vegas provide a surreal experience for first timers and are sure to excite even the most seasoned of veterans. Let’s talk about one of the exciting technologies that are sure to change the world as we know it, or at least the businesses we are familiar with – Amazon Machine Learning.
Introduced on April 9, 2015, Amazon Machine Learning (ML) has received a surge of attention in recent years given its capability to provide highly reliable and accurate predictions with a large dataset. From using Amazon ML to track next generation stats in the NFL, to analyzing real time race data in Formula 1, to enhancing fraud detection at Capital One, ML is changing the way we share experiences and interact with the world around us.
During re:Invent 2018, AWS made it clear that ML is here to stay and has announced many offerings that support development of ML solutions or services. But you may be wondering: What exactly is Amazon ML?
According to AWS’s definition:
We, as a society, are at the point where machines are actively providing decisions for many of our day-to-day interactions with the world. If you’ve ever shopped as a Prime member on Amazon.com, you have already experienced an ML algorithm that is in tune with your buying preferences.
In our Engineer’s Corner, our very own Kris Brandt Amazon Web Service As A Data Lake, discusses the critical initial step towards implementing an ML project, Data Lake creation. In this blog, Kris explores what a Data Lake is and provides some variations to its implementation. The development of a robust data lake is requisite for implementing an ML project that provides the business value expected from the service capabilities. ML runs on data and having plenty of it provides a foundation for an exceptional outcome.
Utilizing existing data repositories, we can work with business leaders to develop those cases for leveraging the data and the ML for strategic growth. You can connect with the Effectual team by emailing [email protected].
Because of ML’s proliferation throughout the market, AWS announced these ML solution opportunities during re:Invent 2018:
AWS Lake Formation “This fully managed service will help you build, secure, and manage a data lake,” according to AWS. It allows you to point it at your data sources, crawl the sources, and pull the data into Amazon Simple Storage Service (S3). “Lake Formation uses Machine Learning to identify and de-duplicate data and performs format changes to accelerate analytical processing. You will also be able to define and centrally manage consistent security policies across your data lake and the services that you use to analyze and process the data,” says AWS.
Amazon Textract “This Optical Character Recognition (OCR) service will help you to extract text and data from virtually any document. Powered by Machine Learning, it will identify bounding boxes, detect key-value pairs, and make sense of tables, while eliminating manual effort and lowering your document-processing costs,” according to AWS.
Data transfer/migration is a key consideration in any organization’s decision to move into the cloud.
If a sound strategy is applied, migration of on-premise data to the cloud is usually a seamless process. When an organization fails to do so, however, it risks running into challenges stemming from deficiencies in technical resources, inadequate planning, and/or incompatibility with legacy systems, to name a few.
Data transfer via AWS Snowball is no exception. If performed incorrectly or out of order, some of the seemingly insignificant tasks related to the data migration process can become substantial obstacles that adversely affects a timeline. The AWS Snowball device can be simple to use if one is familiar with other AWS data transfer services and/or follows all of the steps provided in the AWS Snowball User Guide.However, neglecting a single step can greatly encumber an otherwise ordinary data transfer process.
According to AWS on its service:
When preparing to migrate data from on-premises storage into AWS via a Snowball device, an organization should be aware of the importance of 11 easily overlooked tasks and considerations associated with planning for the data move. They are as follows:
1. Understanding the specifics of the data being moved to the cloud.
Ensure that it is compatible and can transfer seamlessly to the cloud via AWS Snowball. Follow a cloud migration model to help layout specific details and avoid surprises during the data transfer process.
2. Verifying and validating the amount of data being transferred.
Snowball is intended for large data transfers (over 10 terabytes). Using it for smaller data transfers is not a cost-effective option.
3. Verifying that the workstation meets the minimum requirement for the data transfer.
It should have a 16-core processor, 16 MB of RAM, and a RJ45 or SPF+ network connection.
4. Performing a data transfer test on the workstation an organization plans to use to complete the task.
This will not only equip the organization with an understanding of the amount of time needed to perform the transfer, but will provide an opportunity to try various methods of transferring data. Additionally, it will assist with estimating the time the Snowball device will need to be in the organization’s possession, as well as its associated cost.
NOTE: The Snowball Client must be downloaded and installed before this step is performed.
5. Creating a specific administrative IAM user account for the data transfer process via the management console.
This account will be used to order, track, create and manage Snowball Import/Export jobs and return the device to AWS.
NOTE: It is important to avoid using personal IAM user accounts if individuals will be responsible for ordering the device and performing the data transfer.
It is also important to confirm that the selected S3 bucket name aligns with the expectations of the stakeholders.
7. Confirming the point of contact/s and shipping address for the Snowball device.
This is especially important if the individual ordering the device is different from the one performing the data transfer.
8. Setting up SNS notifications to help track the stages of the snowball job.
This will keep the stakeholders informed of the shipping status and the importing of data to the S3 bucket.
9. Being aware of how holidays could affect the progress or process of the data-transfer timeline.
This is important because additional costs are accrued 10 days after the Snowball is delivered.
10. Considering the organization’s administrative processes that might hinder or delay the data transfer process.
By factoring in internal processes (e.g., Change Request management, stakeholder buy-in, technical change moratoriums, etc.) into the timeframe it will take to receive the device, start the job, and ship it back to AWS can help prevent unnecessary fees.
NOTE: The Snowball device has no additional cost if it is returned within 10 days from the date it is received. Following that time, however, a daily late fee of $15 is applied until the date AWS receives it.
11. Keeping the original source data intact till the data import is confirmed.
It is very important that source data remain intact until the Snowball device has been returned to AWS, the data import has been completed, and the customer has validated the data in the S3 bucket(s).
Transferring data from on-premises to an AWS Snowball can be an uneventful endeavor when thorough planning is done in advance of ordering the device. Taking these 11 planning tasks and considerations into account are essential to eliminating some of the potential headaches and stress occasionally associated with this type of activity.
During the past nine years of delivering cloud solutions to government and industry, our team has identified a gap in the delivery of Managed Service Provider (MSP) solutions in a cloud environment.
As an AWS Premier Consulting Partner with the DevOps, Government, and Non-Profit competencies, as well as the GovCloud Skill Partner Status, there are few workloads that we haven’t helped customers migrate and manage in a cloud environment. We also deliver these solutions through Microsoft Azure, where we are a Silver Cloud Partner.
Our focus with MSP is around a structured, repeatable, five-step MSP process
We previously discussed the Evaluation, Automation, and Optimization phases as part of the MSP process. In this blog – the fourth in a series of five on Next Generation MSP (NG-MSP) – we’ll touch on Monitoring, a critical component of a successful MSP strategy from a visibility perspective. Appropriate monitoring of infrastructure as part of our NG-MSP strategy affords both Effectual and our customers the ability to be proactive in measures by collecting, analyzing, and acting on information efficiently.
Further, as we develop a custom NG-MSP playbook for our customer, we’re going to look at where we can integrate with existing Security or Network Operations Center. For us, we don’t see NG-MSP as a one-size fits all model, which is a spot where many traditional MSP providers, even cloud MSP providers, miss the mark.
Proper cloud-first, NG-MSP monitoring must focus on
Adherence to cost and best practices, including tooling from industry leaders such as CloudCheckr
The proper configuration and performance of services such as AWS’s CloudWatch
Incorporating logging tools such as AWS CloudTrail
Analyzing logs through native cloud services such as AWS’s Macie or third-party tools such as Splunk.
During the past nine years of delivering cloud solutions to government and industry, our team has identified a gap in the delivery of Managed Service Provider (MSP) solutions in a cloud environment.
As an AWS Premier Consulting Partner with the DevOps, Government, and Non-Profit competencies, as well as the GovCloud Skill Partner Status, there are few workloads that we haven’t helped customers migrate and manage in a cloud environment. We also deliver these solutions through Microsoft Azure, where we are a Silver Cloud Partner.
Our focus with MSP is around a structured, repeatable, five-step MSP process
We previously discussed the Evaluation and Automation phases as part of the MSP process. In this blog – the third in a series of five on Next Generation MSP (NG-MSP) – we’ll touch on Optimization, a critical component of a successful MSP strategy from both performances as well as a cost perspective. Optimization considers leftover infrastructure from “lift and shift” migration models and keeping abreast of best practices in instance usage, including right-sizing and auto-scaling.
Infrastructure alone isn’t always the full story as licensing plays a large driver in business cases given existing investments or necessary upcoming requirements. This ability to go beyond infrastructure and into a holistic optimization discussion is an area where Effectual continues to provide value. NG-MSP may include an ability to transition from licensed databases to license-included database technologies. Cost savings from reducing or eliminating redundant storage in favor of fault-tolerant architectures is another step forward from a performance and fiscal perspective. It’s also an opportunity to evaluate deployment models to take advantage of microservices.
The cloud-first NG-MSP optimization component continues to focus on
Minimal cloud footprint to meet business requirements
Proper scaling architectures to serve growth and demand
Microservice utilization where appropriate
Built-in security monitoring through tools such as CloudCheckr
Custom NG-MSP delivery models to meet organizational and compliance requirements
Change management through scripting to ensure efficient deployment
We’ve done this long enough to know that NG-MSP models cannot be a one-size fits all proposition, which is why optimizing as much as possible within an environment will allow efficient operation from both a performance and cost perspective.
We stand ready to support your organization’s cloud environment with our NG-MSP services. To discuss options, please reach out to [email protected].
During the past nine years of delivering cloud, our team has identified a gap in the delivery of MSP solutions in the cloud.
As an AWS Premier Consulting Partner with the DevOps, Government, and Non-Profit competencies, and GovCloud Skill Partner Status, there are few types of workloads we haven’t helped customers migrate and manage in a cloud environment.
Our focus with MSP is around a structured, repeatable, five-step MSP process
We previously discussed the Evaluation phase of the MSP process. In this blog – the second in a series of five on Next Generation MSP (NG-MSP) – we’ll touch on the Automation, a critical component of a successful MSP strategy. Automation should include operational workflows; infrastructure provisioning; and, patching and change management.
The cloud-first MSP automation component is focused on
Deploy Infrastructure as Code (IaC)
Establish and secure edge connectivity
Deploy Virtual Private Clouds (VPCs)
Deploy networking / security groups
Set-up application and data tiers
Move applications with scripted builds
Where applicable, it is also important that required regulatory standards are automated when possible to ensure legal compliance in relation to sensitive areas (e.g. SCI, HIPAA, NIST, SOC, etc.). In addition, Continuous Integration/Continuous Deployment (CI/CD) pipelines should be developed and implemented to support reliable delivery of code changes.
MSP models cannot be a one-size fits all proposition. Automating as much as possible within an environment will help offload some of that ongoing MSP administration, allowing those outlier components unique to different organizations to be a focus of the Effectual team.
We stand ready to support your organizations’ cloud environment with our MSP services. To discuss options, please reach out to [email protected].
Cloud-based Managed Service Providers (MSP) changed the Managed Services landscape as we know it.
Unlike traditional on premise-based MSPs, a cloud MSP offers an unmatched level of efficiency, scalability and potential for innovation. This improves an organization’s operations significantly. Through the optimization of processes, automation of routine tasks, monitoring of vital components and integration of simpler workflows, resources can be free to focus on what is most important for the organization.
Our focus with MSP is around a structured, repeatable, five-step MSP process
This blog is the first in a five-part series related to our delivery of Next Generation MSP (NG-MSP).
To get an agency or organization started with Effectual’s MSP program, it is essential for a proper evaluation; specifically, an assessment of the security and network environments and general cloud readiness. Assessing these areas before any MSP activities commence helps identify potential security vulnerabilities and network performance issues. It promotes validation of existing technology, people, and processes while establishing a well-defined reference point.
Our evaluation focuses on the following
Identify apps to move
Develop project schedule
Refine budget / cost optimization
MSP models cannot be a one-size fits all proposition. Automating as much as possible within an environment will help offload some of that ongoing MSP administration, allowing those outlier components unique to different organizations to be a focus of the Effectual team.
We stand ready to support your organizations’ cloud environment with our MSP services. To discuss options, please reach out to [email protected].
“Cloud,” “Machine Learning,” “Serverless,” “DevOps,” – technical terms utilized as buzzwords by marketing to get people excited, interested, and invested in the world of cloud architecture.
And now we have a new one – “Data Lake.” So, what is it? Why do we care? And how are lakes better than rivers and oceans? For one, it might be harder to get swept away by the current in a lake (literally, not metaphorically).
A Data Lake is a place where data is stored regardless of type – structured or unstructured. That data can then have analytics or queries ran against them. An allegory to a data lake is the internet itself. The internet, by design, is a bunch of servers labeled by IP addresses for them to communicate with each other. Search Engine web crawlers visit websites associated with these servers, accumulating data that can then be analyzed with complex algorithms. The results allow a person to type in a few words into a Search Engine and receive the most relatable information. This type of indiscriminate data accumulation and the presentation of context-relatable results is the goal of data lake utilization.
However, for anyone who wants to manage and present data in such a manner, they first need a data store to create their data lake. A prime example of such a store is Amazon S3 (Simple Storage Service) where documents, images, files, and other objects are stored indiscriminately. Have logs from servers and services from your cloud environments? Dump them here. Do you have documentation that are related to one subject, but are in different formats? Place them in S3. The filetype does not really matter for a data lake.
ElasticSearch can load data from S3, indexing your data through algorithms you define and providing ways to read and access that data with your own queries. It is a service designed to provide customers with search capability without the need to build your own searching algorithms.
Athena is a “serverless interactive query service.” What does this mean? It means, I can load countless CSVs into S3 buckets and have Athena return queried data as a data table output. Think database queries without the database server. Practically, you would need to implement cost management techniques (such as data partitioning) to limit the ingestion costs per query as you are charged on the amount of data read in a query.
Macie is an AWS service that ingests logs and content from all over AWS and analyzes that data for security risks. From personal identity information in S3 buckets to high risk IAM Users, Macie is an example of what types of analysis and visualization you can do when you have a data lake.
These are just some examples on how to augment your data in the cloud. S3, by itself, is already a data lake – ‘infinite’, unorganized, and unstructured data storage. And the service already is hooked into numerous other AWS services. Data lake is here to stay and is a mere stepping stone to utilizing the full suite of technologies available now and in the future. Start with S3, add your data files, and use Lambda, ElasticSearch, Athena, and traditional web pages to display the results of those services. No servers, no OS configurations or security concerns; just development of queries, lambda functions, API calls, and data presentation – serverless.
Our team is building and managing data lakes and the associated capabilities for multiple organizations and can help yours as well. Reach out to our team at [email protected] for some initial discovery.
The Right Partner is Better than a Crystal Ball (7 of 7)
Mistakes can create amazing learning opportunities and have even led to some of the most beneficial discoveries in human history, but they can also have far-reaching, time-consuming, and costly implications.
Luckily, someone else has probably already made the mistakes you’re bound to make, so why not reap the benefits of their errors and subsequent experience?
Know Your Strengths and Limitations
One of my passions in life is building things. Working with my hands to create something new and unique fills me with a sense of accomplishment beyond description. Over the years, I’ve taken on a variety of projects to scratch this itch; from baking, to renovating my kitchen (I’m really quite proud of the custom-built cabinets I made from scratch), to customizing various vehicles I’ve owned over the years. Along the way, I’ve built several Jeeps into serious off-road machines.
In the years before YouTube, when I was first developing the skills required to lift and modify a Jeep, I often encountered situations where I wasn’t confident in my knowledge or abilities. I knew that the vehicle I was working on would need to be just as safe and reliable on the freeway as it would be out in the middle of nowhere – places where the results of my efforts would be stress tested and the results of poor workmanship could be catastrophic. Each time I encountered an area where I had limited knowledge or experience, I would do all the research I could, then find someone with the right experience who could coach me through the process (and critique my work along the way).
Fortunately, I had a ready supply of trusted friends to advise me. In my time spent driving these heavily modified vehicles, I did encounter the occasional failure, but thanks to the skills I developed under the direction of these watchful eyes, none of them ever put me at significant risk.
As enterprises modernize their infrastructure, they should look at IT as a contributor to business innovation rather than a means to an end. Alongside this shift in view, the expertise required to define strategy, architect solutions, and deliver successful outcomes is increasingly difficult to acquire. In this dearth of available talent, many of the enterprises I’ve been dealing with are struggling with the decision between:
Delaying modernization efforts
Plowing ahead and relying on internal resources to get trained up and tackle the challenges
Bringing in a third party to perform the work of modernization.
Unfortunately, none of these options is ideal.
Choosing the Best Path
The first option, delaying modernization, limits the enterprise’s ability to deliver products and services to their stakeholders and clients in innovative ways – opening the door for disruptive competitors to supplant them. For dramatic evidence of this, look at the contrasting stories of Sears, the first company to offer ‘shop at home’ functionality, and Amazon.com, the disruptor who has supplanted them as the go-to home shopping solution. The option of delaying presents significant risk, but the risk of assigning internal resources to address issues they’re not fully prepared to handle should not be underestimated.
Plowing ahead with a team that’s facing unique challenges for the first time means you’ll be lacking the benefits of experience and hindsight. In my previous posts I’ve discussed some of the hidden traps encountered along the modernization journey, many of which can only be seen in hindsight. These traps can only be avoided once you’ve had the inevitable misfortune and experience of having fallen into them before. “Fool me once…”
It’s similar to the Isla de Muerta from the Pirates of the Caribbean movies; “…an island that cannot be found – except by those who already know where it is.” Unlike the movie, there’s nothing magical about the pitfalls that litter the path to modernization. Many of the concepts that we have accepted as IT facts for decades are invalidated by modern approaches. So, the logical decision seems simple: outsource the effort to someone who has been there before.
Finding the Right Experience and Approach
Bringing in a partner that is experienced in the many areas of IT modernization is the safest of the three options, but your mode of engagement with this third party directly relates to the benefits your enterprise will enjoy. The best advice I can offer is to look for a provider who views modernization from a business perspective, not merely as a technical effort. Most providers will tout their technical expertise (we at effectual do too), but the reality is that nearly all competent providers have people with the same certifications. Technical certifications are no longer the differentiator they used to be. When you are interviewing these teams, ask how they plan to interact across your various enterprise teams outside of IT. If they look puzzled, or don’t have an answer, you know that they are not a business solution provider.
Once you have an idea around who is able to help with your modernization efforts, you need to make a decision regarding the methodology that will best suit your enterprise. One possible route is to completely turn the effort over to the outsourced team. While this is a fairly risk-free approach that leaves you with transformed IT when the project is over, you don’t gain any of the expertise required to manage your environment moving forward. I’ve found that the greatest benefits are realized when an enterprise and their provider partner together on the solution.
Providers as Partners Partner resources collaborate with enterprise resources to deliver solutions, while also providing training, insight, oversight, and guidance.
In this scenario, the partner team takes the lead on the migration project under the executive direction of the enterprise, just like my friends who would help with my vehicle builds. Partner resources collaborate with enterprise resources to deliver solutions, while also providing training, insight, oversight, and guidance. At the end of the day the enterprise enjoys a better purpose-built solution and develops the expertise to enhance it as additional business requirements are identified, or existing requirements evolve.
What the Future Holds
As modernization efforts start to take hold and your teams gain confidence, you should not consider the journey complete. This is the point in the revolution where a modernized organization can truly view IT and engineering as the linchpin of your competitive advantage, whether it be through cloud adoption, big data, artificial intelligence, mobility, or other current technologies. Historically, the interaction between business and IT has been a two-step process. The business conceptualizes features that would benefit some constituency, whether it be internal or external, then directs IT to build it.
In the new world, where technological capabilities are rapidly evolving and growing, the competitive advantage comes primarily from changing that two-step process. The business first asks IT “What is possible?” and then business teams collaborate with IT to deliver forward‑thinking solutions. This is the behavior that enables innovation and disruption within industries. We’ll explore this topic in depth in a future post.
Learning from the Experts
What has made me a successful builder over the years has been my good fortune to have skilled artisans available to guide and coach me through the work as I was learning how to do it. As I learn tips and tricks from experts, I begin to behave like an expert and deliver high quality work. As you look to your modernization efforts your enterprise can Cloud ConfidentlyTM and see similar growth by bringing in the right partners to help lever your team’s skills and understanding.
The introduction of new technologies impacts workflow, changes the way your teams go about doing their jobs, and how they communicate with each other and customers. Your teams need to understand the day-to-day value of transformation, and they need to feel like part of the process.
Preparing your teams for potentially radical culture changes are especially critical for financial services organizations, which have historically been hesitant to adopt new and innovative technologies due to the heavily regulated nature of the FinServ industry.
Technical Preparedness: It’s common for IT teams to feel a loss of control due to cloud transformation. John Dodge of CIO has said that in-house IT can start feeling like a service broker without a sense of ownership. IT will confront a learning curve with interfaces, APIs, and provider management. Technical training and growth must be prioritized.
Skills Building: IT isn’t the only department that will have to acquire new skills. Technical skills are a given, and will be increasingly vital, but your team will likely also need to learn new project management skills and develop a keen understanding of the new realities of security and compliance in the cloud.
Frequent Communication and Transparency: Some IT departments can be very resistant to change. During a cloud transformation, there can be tension when your IT team is used to getting under the hood and having total control. Frequent, transparent communication, however, can mitigate resistance in IT. A simple email won’t do the trick. Your IT team needs to know what’s happening – and, more importantly, why— far in advance of it happening. Transparent communication can and should be part of a positive feedback loop that informs the track of on-going training.
Considerations in Cloud Transformation
Proper planning enables a successful project. Here are some key considerations to discuss internally and with your partners:
Compliance and Security: This is at the top of the list for a reason – third-party and government-mandated security requirements for financial services companies leave little to no room for error. Your organization, partners, and cloud service provider must all understand your regulatory and compliance requirements and address your overall cloud security posture so that you can maintain compliance. This is literally Job One.
Performance: Performance is of key importance to financial services, who frequently need high performance compute to power their transactions and data analysis. You need easily deployable, easily scalable high-performance processing resources for simulations and modelling, machine learning, financial analysis, and data transformations. This requires a detailed ROI and financial analysis to understand costs and avoid sticker shock.
Intellectual Property: What is more important to financial services than their intellectual property? Putting that anywhere outside of your own systems is a hugerisk, but a properly architected cloud solution can ensure that your data is safer in the cloud than it is in legacy solutions.
Transitioning to the cloud can give your business a competitive edge. For financial services lacking large, experienced in-house IT teams, it’s worth considering a partner and leveraging their expertise to make your transition a success.
What I Saw at AWS re:Invent That You May Have Missed
It was an interesting re:Invent for me – launching our new company, the constant feeds from AWS and several partners filled with litanies of new feature and product announcements, multiple multi-hour keynote speeches (yes, I did sit through all of them), working to discern hype from reality, fighting through the throngs of people, and trying to convince the 12-year-old child inside me that I don’t need a Deep Racer.
In the midst of all of this, there were a couple of interesting success stories that really validated my recent lines of thought regarding the business ramifications of cloud transformation.
Andy Jassy’s wide-ranging keynote included product and feature announcements related to Database, Storage, Machine Learning, Security, Blockchain, and who can forget Outposts (so you can run the cloud in your closet).
Revolutionizing the Insurance Industry
In his keynote, Andy Jassy introduces and turns the stage over to Dean Del Vecchio, EVP, CIO and Head of Enterprise Shared Services for Guardian Insurance. Guardian is a great example of a legacy enterprise that has embraced the revolution despite being in a highly regulated industry . With the words, “I’m going to start in a place that may not be expected – I’m going to talk about our workplace Strategy”, Mr. Del Vecchio had my full attention. From there he went on to talk about Guardian’s multi-year transformation that needed to take place before they moved their first workload to the cloud. They changed their office environment, modernized project methodology, trained their staff on new technologies, and ultimately revolutionized their culture. This was not a tale of a headlong sprint to cloud, it was a thoughtful, self-aware, and measured approach.
Dean Del Vecchio, EVP, CIO and Head of Enterprise Shared Services for Guardian Insurance
Guardian is a great example of a legacy enterprise that has embraced the revolution despite being in a highly regulated industry.
Once cultural change had begun to take root, Guardian stood up AWS environments and ran Proof of Concept workloads for a full year before the first workload was moved. They identified gaps during this time, and worked with vendors to develop solutions that enabled Guardian to be bold and confident in their migration. They documented their technological biases, the core being a Cloud First posture (rather than an All-In on cloud directive). Because of their approach and inherent understanding of the revolutionary nature of 21st century technologies, they were able to take a Production First approach to moving to the cloud. There were undoubtedly significant pain points and localized failures throughout this journey, but having come through the bulk of it now, Guardian sees their adoption of cloud as a competitive advantage. They have revolutionized the way they interact with their customers and are excited to begin making use of forward-looking technologies like AI, AR, and VR in the cloud to further enhance client experience.
What was most interesting to me was what their journey was not.It was not exclusively an IT effort, a rapid lift and shift migration, or even primarily technical in nature. It was definitely not shortsighted in nature. It did revolutionize a 158-year-old Fortune 250 enterprise, from top to bottom, and they are now well-positioned for another 158 years of success.
Fender Guitars – re:Invented
In Werner Vogels’ keynote, he started off by talking about minimizing blast radius, one of my favorite topics, and he wrapped up by talking about Deep Racer, the toy that my inner 12-year-old seems to think I need. In between, amongst some explanations around high-level database software design principles, he introduced Ethan Kaplan, the Chief Product Officer, Fender Digital, from Fender Musical Instruments, one of the premier guitar manufacturers in the world. Mr. Kaplan shared some of the results from research Fender conducted around their client base.
Basically, it boils down to three key points:
90% of all first-time guitar buyers quit playing within 6 months of the purchase
Those who don’t quit will purchase 8-10 guitars over their lifetime
Guitar players spend roughly 4x the value of their first guitar on lessons
From this data they realized that there is a significant market for guitar lessons, and if they can find a better way to improve on the traditional lesson model, they will likely sell many more guitars.
Fender Digital Took a Cloud First Approach for All New Application Development
Fender Digital Chief Product Officer, Ethan Kaplan, discussed the apps and teaching methodology created to help new guitarists see success. Now shooting 4K video 6 days a week, Fender worked alongside AWS in developing a full video processing pipeline architecture that brings terabytes of new content to their users every day.
This led them in two directions. They created a number of apps to help new players understand their new instrument, and they developed an app and teaching methodology that is more fun for the new player and helps them see musical success sooner. Combined, these convert a higher percentage of new players into lifetime players. The story I saw here is that the adoption of revolutionary 21stcentury IT technologies has enabled Fender to transform their business. They have always made great guitars and stringed instruments, and will continue to do so, but now their business is more and more about teaching people how to play. Fender is shooting 4k video on two soundstages six days a week to support this new mission, producing Terabytes of content per day. Working alongside AWS to create a video processing pipeline architecture, Fender can now automatically transcode down to cellphones and up to 4k televisions, simultaneously auto-populate their CDN and CMS, and archive raw video to glacier 24 hours a day, 7 days a week. Before long the core of their business will be developing and delivering musical training curricula with the manufacture of instruments being a sideline.
Realizing 21stCentury IT
For the past couple of months, I’ve been writing about how cloud and cloud-related technologies are revolutionary in nature, and how your decision to adopt cloud should be viewed as a business effort rather than a strictly technical effort. These two stories from re:Invent keynotes are great examples of similar success that I have seen with clients who have recognized and acted on this principle.
If you are reading this and you’ve already begun a cloud effort that looks like a data center move, it’s not too late to change course. Don’t fall into the trap of thinking that we’ll just Lift and Shift to the cloud now and transform later – this is a prime example of thinking of cloud as a technical rather than business solution. I’ve yet to see an enterprise be successful at following through on this approach. If you’re not looking at 21stcentury IT as a business change agent and competitive advantage, it won’t be long before one of your competitors is.
The Reality of the Cloud for Financial Services – Part 1
According to a report from Markets and Markets, the financial services cloud market is set to reach a total market of $29.47 billion by 2021, growing 24.4 percent.
The study further found that most of this growth will be right here in North America. At the forefront of this trend, Capital One has made public declarations around their all-in cloud, all AWS strategy. Their significant presence on the expo floor at AWS re:Invent was further evidence of this commitment. The growth of cloud adoption in this sector is driven by a simple truth: Financial services are being transformed by the cloud.
Still, some Financial Services companies struggle with the challenge presented by the maze of legacy services built up over decades of mergers and acquisitions. It’s all too common for financial services organizations to still be using IBM mainframes from the 1960s alongside newer technologies. This can make the prospect of cloud transformation seem daunting to say the least.
However, the question is not if your organization will transition to the cloud, but when. Cloud transformation benefits greatly outweigh the risks — and your competitors are already moving. The longer you wait, the further behind you fall.
Benefits of Cloud Migrations from Legacy Solutions
There are specific benefits to financial services organizations migrating legacy environments to the cloud:
Efficiency: A cloud migration offers opportunity for increased efficiency at decreased operational costs. For financial services organizations looking to revolutionize their IT procurement model, a cloud migration from legacy environments is a quick and easy win, opening the door to more modern tools and services.
Decreased Storage Costs: The regulatory requirements for data retention can create enormous costs for financial services. Moving to AWS cloud storage solutions can significantly reduce costs, while still meeting stringent data security requirements. A well architected cloud storage solution will meet your needs for virtually unlimited scalability while improving cost transparency and predictability.
Increased Agility: To improve their competitive edge, Financial Services organizations are seeking ways to become more agile. A well-planned cloud transformation will result in applications and platforms that effortlessly scale up and down to meet both internal and customer-facing needs. What’s more, a successful cloud transformation means access to new and better tools and advanced resources.
Improved Security: Cloud solutions offer access to enterprise-level equipment and the security that comes along with that. This is normally only within the budget of very large organizations. Cloud services provide increased redundancy of data, even across wide geographical areas. They also offer built-in malware protection and best-in-class encryption capabilities.
In my next post we’ll discuss the “culture transformation” that will enable Financial Services to maximize the return on their cloud transformation investment.
There was a time, not so long ago, when that was the first question a technician would ask when attempting to resolve an issue with a PC or a server that evolved from PCs. This was not limited to servers; IT appliances, network equipment, and other computing devices could all be expected to behave oddly if not regularly rebooted. As enterprise IT departments matured, reboot schedules were developed for equipment as a part of routine preventative maintenance. Initially, IT departments developed policies, procedures, and redundant architectures to minimize the impact of regular reboots on clients. Hardware and O/S manufacturers did their part by addressing most of the issues that caused the need for these reboots, and the practice has gradually faded from memory. While the practice of routine reboots is mostly gone, the architectures, metrics, and SLAs remain.
Five Nines (or 99.999%) availability SLAs became the gold standard for infrastructure, and is assumed in most environments today. As business applications have become more complex, integrated, and distributed, the availability of individual systems supporting them has become increasingly critical. Fault tolerance in application development is not trivial, and in application integration efforts it is orders of magnitude more difficult, particularly when the source code is not available to the team performing the integration. These complex systems are fragile and will behave in unpredictable ways if not shut down and restarted in an orderly fashion. If a single server supporting a piece of a large distributed application fails, it can cause system or data corruption that will take significant time to resolve, impacting client access to applications. The fragile nature of applications makes Five Nines architectures very important. Today, applications hosted in data centers rely on infrastructure and operating systems that are rock solid, never failing, and reliable to a Five Nines standard or better.
As we look at cloud, it’s easy to believe that there is an equivalency between a host in your data center and an instance in the cloud. While the specifications look similar, critical differences exist that often get overlooked. For example, instances in the cloud (as well as all other cloud services) have a significantly lower SLA standard than we are used to, some are even provided on a Best Effort basis. It’s easy to understand why this important difference is missed – the hardware and operating systems we currently place in data centers are designed to meet Five Nines standards, so it is assumed, and nobody asks about it anymore. Cloud-hosted services are designed to resemble systems we deploy to our data centers, and although the various cloud providers out there are clear and honest about their SLAs, they don’t exactly trumpet the difference between traditionally accepted SLAs and those they offer from their rooftops.
A Best Efforts SLA essentially boils down to your vendor promising to do whatever they are willing to do to make your systems available to you. There is no guarantee of uptime, availability or durability of systems, and if a system goes down, you have little or no legal recourse. Of course, it is in the interest of the vendor and their reputation to restore systems as quickly as possible, but they (not you) determine how the outage will be addressed, and how resources will be applied to resolve issues. For example, if the vendor decides that their most senior technicians should not be redirected from other priorities to address the outage, you’ll have more junior technicians handling the issue, who may potentially take longer to resolve it – a situation which is in your vendor’s self-determined best interest, not yours.
There are several instances where a cloud provider will provide an SLA better than the default of Best Efforts. An example of this is AWS S3, where Amazon is proud of their Eleven Nines of data durability. Don’t be confused by this, it is a promise that your data stored there won’t be lost, but not a promise that you’ll be able to access it whenever you want. You can find availability SLAs for several AWS services, but none of them exceed Four Nines. This represents effectively 10x the potential outage time over Five Nines, and applies only to the services provided by the cloud provider, not the infrastructure you use to connect to them or your applications which run on top of them. The nature of a cloud service outage is also different than one that happens in a data center. In your data center, catastrophic all-encompassing outages are rare, and your technicians will typically still have access to systems and data while your users do not. They can work on both restoring services and “Plan B” approaches concurrently. When systems fail in the cloud, often times there is no access for technicians, and the work restoring services cannot begin until the cloud provider has restored access. This typically leads to more application downtime. Additionally, when systems go down in your data center, your teams can typically provide an ETA for restoration and status updates along the way. Cloud providers are notorious for not offering status updates while systems are down, and in some cases, the systems they use to report failures and provide status updates rely on the failed systems themselves – meaning you’ll get no information regarding the outage until it is resolved. Admittedly, these types of events are rare, but the possibility should still give you pause. So, you’ve decided to move your systems to the cloud, and now you’re wondering how you are going to deal with the inevitable outages. There are really only a few options available to you; first, you can do nothing and hope for the best. For some business applications this may be the optimal (although most risky) path. Second, you can design your cloud infrastructure like your data centers have been designed for years. My last two posts explored how expensive this path is, and depending on how you design, it may not offer you the availability that you desire anyway. Third, you can implement cloud infrastructure automation and develop auto-scaling/healing designs that identify outages as they happen and often respond before your team is even aware of a problem. This option is more cost-effective than the second option, but it requires significant upfront capital and its effectiveness requires people well-versed in deploying this type of solution – people who are in high demand and hard to find right now. Finally, the ideal way to handle this this challenge is to rewrite applications software to be cloud-native – modular, fault-tolerant applications that are infrastructure aware, able to self-deploy and self-re-deploy through CI/CD patterns and embedded infrastructure as code. For most enterprise applications this would be a herculean effort and a bridge too far. Over the past several decades, as we’ve made progress in IT towards total availability of services, you’ve come to rely on, take comfort in, and expect your applications/business features to be available all the time. Without proper thought, planning, and an understanding of the revolutionary nature of cloud hosted infrastructure, that availability is likely to take a step backward. Don’t be like so many others and pay a premium for lower uptime. Be aware that there are hazards out there and bring in experienced people to help you identify the risks and mitigate them. You’re looking for people who view your moves toward the cloud as a business effort, not merely a technical one. Understand the challenges that lie ahead, make informed decisions regarding the future of your cloud estate, and above all, Cloud ConfidentlyTM!
Don’t Take Availability for Granted Over the past several decades, as we’ve made progress in IT towards total availability of services, you’ve come to rely on, take comfort in, and expect your applications/business features to be available all the time. Without proper thought, planning, and an understanding of the revolutionary nature of cloud hosted infrastructure, that availability is likely to take a step backward. Bring in experienced people to help you identify the risks and mitigate them.
DevOps methodology requires seeing engineering and IT more holistically. They work together throughout the product lifecycle, with some organizations merging them into a single team.
DevOps culture decreases time to market, while increasing reliability, scale, and security through closer collaboration between development (engineering) and operations (IT). It can require a sea change in your company’s culture, but again — the costs of not adopting DevOps can be far greater.
DevOps methodology requires seeing engineering and IT more holistically.
Fostering Communication and Cooperation
Create a unified reporting structure for DevOps teams. Set clear expectations and guidelines, driven by fundamental values.
Build cross-functional teams including both engineers and IT technicians.
Leverage engineers to assist IT technicians in understanding complexities involved in delivering business features.
Leverage IT technicians to help engineers understand how involved 24/7 support of those features is.
Automate repetitive tasks, freeing up engineers and IT technicians to perform more creative, value-adding work while reducing human error.
Make feedback a fundamental feature of workflow to encourage communication-driven action. Create a culture of data sharing.
Driving Improvements through DevOps
While DevOps is generally part of product development, it can also be an invaluable tool during cloud migration. No less an authority than Bina Khimani, Global Head of Partner Ecosystem (Cloud Migrations) at Amazon Web Services, made this case in late 2017. A survey of 450 C-level and VP/director-level executives in the United States, Canada, and the UK found 54 percent already leveraged DevOps methodologies.
This is not an easy direction to move in, even with the best circumstances. It requires unwavering leadership in the truest sense of the word. However, the rewards are wide-ranging, including:
Continuous development with shorter development timetables.
Faster problem resolution.
More productive teams with higher morale and engagement.
More opportunities for professional development.
More time spent creating and innovating, as opposed to fixing and maintaining.
Cloud deployment can be the perfect time to start adopting these changes in your organizational culture. A successful adoption means that not only does everyone win, but everyone will feel like they’ve won as well. That’s a value you just can’t put a price on.
A conscious and concerted effort is required to get everyone on the same page, pulling in the same direction.
Complementary Missions IT and Engineering have different purposes, but their mission statements can and should complement one another, rather than causing them to butt heads. Involve both IT and Engineering team members when crafting mission statements to make sure that one team doesn’t dominate the other. The purpose here is gathering their input and voice to create mutually beneficial missions.
Open Communication You need to encourage open, clear, and direct communication between departments, especially during disruptive periods. There’s nothing wrong with formalizing this diplomatic process. Having regular meetings where the two departments can hash out disagreements and sticking points can be worth its weight in gold, helping to diminish or eliminate tension altogether. Remember the Agile principle that face-to-face communication is the best.
Complementary Needs Emphasize points of agreement. Where they don’t exist, try to find needs and wants that are complementary. When it’s not possible to make everyone win, it might be possible for each department to feel like they’ve won enough.
Cooperative Culture Create a culture where one department sees the victories of another as successes by fostering a workplace where departments see themselves as part of a greater holistic whole rather than competing factions. Encourage self-organization in keeping with Agile principles.
Cloud migration is about more than just “lifting and shifting” applications and data or modernizing applications for the cloud, it’s about transforming the culture of your entire organization.
Cloud migration is about transforming the culture of your entire organization.
How a Cloud Migration Can Inspire a DevOps Culture
You can use your cloud migration as an opportunity to implement Agile development principles. Defusing tension is an important element of a successful cloud migration. Once you have the broader strokes squared away, you can start drilling down into the specifics.
Automation There’s at least one thing both IT and DevOps immediately have in common: they both want to identify and address manual, inefficient, error-prone tasks. Automating these tasks can free up engineering and IT to focus on more creative and value-adding work. Embracing automation is often the way to gain cross-departmental wins that increase cooperation and morale amongst these teams.
Framing How you frame a cloud migration can go a long way toward garnering goodwill from both departments. Cloud migrations can be framed as the inevitable march of progress, something we all have to “deal with,” or they can be framed as a challenge requiring input from both departments. The former is a quick way to create resentment, but the latter is a valuable way to inspire the best in your people, including healthy competition and creative cooperation.
In my next post, I’ll explore the adoption of DevOps methodology, which sees engineering and IT not as opposing factions, but two parts of a greater whole.
One of the most difficult challenges facing businesses today is bridging the “Skills Gap” in IT.
This is a phenomenon whereby a business’s desire to leverage new and expanding technologies is hindered by the lack of available talent and skillsets to architect, implement, and manage these technologies.
Businesses commonly navigate this gap through the use of outside consultants, relying on a team of high-performing individuals holding extensive experience within a specific domain. Engaging a consultancy to assist in the architecture and implementation of desired technologies accelerates adoption and provides breathing room for a company’s own staff to learn and practice the new technologies. This time to practice and learn should not to be overlooked and is essential to the success of any transformative process. The “Building a car while driving” metaphor comes to mind.
This skills gap is not limited to businesses looking to implement new technologies. Ironically, this conundrum has not bypassed IT vendors, and it is fairly common to see the same gap between IT Sales Professionals and the technologies they’re selling.
It is fairly common to see a gap between IT Sales Professionals and the technologies they’re selling.
The Team Approach
IT vendors separate the necessary skills to govern a sales cycle (commercial skills) with the knowledge and expertise of the products they’re selling (technical skills) by partnering a commercially skilled employee with a technically skilled employee. This approach has permeated the market to such an extent that prospective clients now expect their vendors to show up Noah’s Ark style, two by two.
There is still a struggle to find technically skilled people to accompany commercially skilled sales staff. Mitigation strategies have a certain effectiveness; teaming four or five commercially skilled individuals with a single technically skilled employee allows IT vendors to maximize the effectiveness of their expertise. Technically skilled people are not super-human, although some of the most brilliant deserve to be classified as such, and they can only support so many client conversations, differing sales cycles, and solutions. Eventually “Something’s Gotta Give”. This mitigation is merely a stop-gap solution. The underlying issue is still prevalent; all businesses need a certain number of technically skilled individuals to remain competitive. The fight for talent continues.
IT vendors are also trying to address this gap by putting a ‘paywall’ between prospective clients and technically skilled individuals, charging clients for their employees’ time and experience to assess their current needs. This solution lines up neatly with the consultancy approach. If you have something a business wants, don’t give it away for free. This also places more responsibility and pressure on the commercially skilled individual, forcing them ensure they qualify a true need with their clients before requesting a potentially limited resource. When a client opts to pay for the time and experience of a technically skilled individual, they are showing real intent to form a business relationship.
This approach fails when commercially skilled individuals don’t qualify the needs of a client in detail and overpromise on the capabilities of their technical counterparts. Again, these technologists are not superhuman and come with a caveat – ‘Magic Wand Not Included.’
Without a technically skilled individual in the room when qualifying questions are being asked, commercially skilled individuals can be forgiven for misunderstanding the technical requirements of a client. In these scenarios, sales may inadvertently offer a solution to a client need that their product or service does not solve. This can lead to perpetuating the classic salesperson stereotype– just agree to anything in order to close a deal.
At effectual, our approach to sales, which we believe is shared by the majority of our peers, is to avoid this at all costs. At no point do we want to have a client conversation that starts with “But you said…”. This stems from a desire to do business openly, honestly, and with integrity. Recognizing that not every client is in need of what you are selling is vital.
The Importance of Professional Development
With this context, I’d like to address the crux of how I personally went about bridging the gap. In order to avoid overpromising, underdelivering, and starting a client relationship that is doomed from day one, it is my belief that it is now the responsibility of a commercially skilled individual to gain some measure of technical skill. In fact, I’ll phrase that more strongly: gain as much technical skill as necessary so you can speak with confidence or reply with “I don’t know but I’ll find out.”
Commercially skilled individuals may have raised their eyebrows, adopted an incredulous look, or stopped reading altogether, and that’s fine. Nowhere in a commercially skilled employee’s job description does it state they need to be technically qualified or expected to perform two job roles for a single salary. There is a logical line of thinking that emerges here. If they were hired for the skills they have already acquired, why would should they be expected to develop skills in a different discipline for a role they have no intention of performing.
My argument however is simple: why not? Broadening your horizons is never a negative, and learning new skills, gaining empathy for other people’s challenges, and respecting their achievements is never a bad thing. Meeting your current and potential clients halfway or the whole way is never a bad thing.
With technically skilled individuals being in such high demand, and without enough of them to go around, surely being able to operate without them creates a competitive advantage. I’m not suggesting that the commercially skilled pivot entirely and a embark on a drastic career change, only that a little knowledge is empowering, and a lot of knowledge is powerful. Watch as the atmosphere of client meetings change from “I’m being sold to” to “this person knows what they’re talking about.”
You can gain a better understanding of exactly how a product or service can assist a prospective client, identify incompatibility early on, and qualify out with confidence. It takes an expanded skill set to understand when the fit is not right, and it takes integrity to step away.
Earning the Respect of Your Clients and Your Team
Technically skilled individuals who play the yin to commercial individual’s yang will be Sales’ biggest supporters. I know this from experience and have made some wonderful friendships as a result (I hope they’re reading this). Operating with more autonomy doesn’t mean you are trying to replace them, just helping to ease their burden. When you finally do request help, they know it will be an interesting challenge, an opportunity for them to impart some knowledge on someone keen to absorb it, or to get creative with a solution because the standard approaches aren’t working.
How do they know all this? Because they know that when you ask, you’ve already ruled out the most common technical approaches. You’ve met them halfway.
I don’t presume to speak on behalf of my better qualified and more experienced technical counterparts, but I hope they’ll agree that they get the biggest kicks out of solving tough challenges, not answering the same questions over and over.
Yes, I feel it’s important for commercially skilled individuals to bridge the gap between commercial and technical skills. Selfishly, it’s for their own benefit for now, but it’ll soon become a necessity as the market-wide skills gap continues to grow.
Tom Spalding is a Strategic Account Manager at effectual, Inc.
But it’s not unusual for the team supporting your digital infrastructure (IT) and your team developing new applications and refining legacy apps (software engineering) to exist in a state of tension at the best of times. Cloud migrations can heighten this tension to the extreme.
The needs and wants of Engineering and IT often come into conflict during a cloud migration. What’s more, the heightened tension can linger around long after the migration is done. That’s not just unpleasant. It can seriously impact your bottom line. So how do you mitigate the tensions between IT and Software Engineering?
Embracing DevOps methodology is a good way to move toward a world where Engineering and IT are seen as two parts of a more cohesive whole. This might require significant change in your organization. But the cost of complacency can be much higher.
Embracing DevOps methodology is a good way to move toward a world where Engineering and IT are seen as two parts of a more cohesive whole.
Understand IT Tension
Common causes of IT tension include:
Conflicting Missions The distinct missions of IT and Engineering can put them instantly into conflict. Engineering is focused on innovating and getting new applications out on time. IT is focused on providing a stable and secure environment for everyone from accounting to the front desk. So, while Engineering is one of many balanced priorities for IT, they believe they should be the top priority. This belief isn’t entirely unjustified. Satisfying clients and developing software are first on the list of Agile Methodology for a reason.
Lack of Communication and Understanding Different missions, scopes, and priorities can make communication difficult even during the best of times. In some cases, there may not even be enough of a shared language for communication. Disputes between IT and DevOps are often communicated and “negotiated” through third parties, adding further opportunity for creating tension. Lack of communication frequently leads to the blame game when things go wrong. This, in turn, deepens mistrust and animosity between the two departments.
Interdepartmental animosity between IT and Engineering doesn’t have to be a fact of life. Management can take steps to move two adversarial departments onto the same page. There might still be rivalries, but that can be healthy. Rivalries that are tempered by mutual respect inspire harder work and greater creativity. Organizations capable of this level of honesty, self-awareness and accountability, will gain a significant competitive advantage over those that are not.
This is the first post in a series of three that discusses the benefits of moving to a DevOps methodology. Next time (Defusing the Tension) we’ll explore some of the ways DevOps can address and defuse tensions within IT departments.
Cloud: The Mirage of Massive Cost Savings (Ketchup on the side) (5 of 7)
“Why are you moving to the cloud?” is a question I’ve asked more times than I can count. It’s one of the first questions posed to a potential client, for multiple reasons.
The two most important reasons are; first, I want to get a little insight into how thoughtful/educated this potential client is in relation cloud, and second, I want to understand what metrics will be used to determine success and or failure of the project we are considering undertaking. Potential clients respond to this question in various ways, but almost always, one of their first answers is around saving money and/or cutting costs. When I hear this response, I ask a couple of follow-up questions to clarify how they plan on accomplishing this ambiguous goal. More often than not, they have no idea how they will recognize cost savings and many just expect it to be a natural benefit of moving their VMs to the cloud. This near universal acceptance of a broad notion, with little factual basis, reminds me of the story of ketchup. In the mid 19thcentury, a doctor took the ketchup of the time (which was basically fermented mushroom sauce or ground up fish innards –further reading on this if you are so inclined ) and added tomatoes to it. He made some somewhat dubious claims regarding the maladies that could be cured by his new ketchup, which were picked up by the press. By the later part of the 19thcentury, with the help of unscrupulous hucksters along the way, nearly everyone believed that ketchup cured all ills. While ketchup does have some definite health benefits and is a very tasty condiment; it’s rich in Vitamin C and anti-oxidants; a cure-all it is most definitely not.
The truth is, simple cloud migration, even when instances are right sized and Reserved Instances (RIs) are purchased, is unlikely to produce significant cost efficiency for infrastructure that isn’t properly architected to take advantage of cloud services. In my last post I shared an example of two different cloud deployment strategies for a sample application. The five-year total operating costs were roughly $350k for one strategy and $14k for the other. The difference; to recognize the operational cost savings of $336k over 5 years, an enterprise would need to spend roughly $100k and several months of effort upfront. Enterprises are wary of the upfront costs, 125% of the expensive model’s projected first year operating costs, or 571% of those first quarter operating costs, so they make a short-term financial decision to proceed with the $350k option. More often than not, this decision is made in the IT department, based on their limited budget visibility, not at the executive level where greater budgetary visibility and enterprise strategy is handled. Another dirty little secret about cloud cost management that doesn’t often percolate up to executive levels is that the flexibility of cloud allows your IT teams to immediately spin up services that generate significant cost with little or no financial oversight until the invoice comes due. For example, common compute instances at AWS can cost from $6 – $10 per hour with specialized services available through the marketplace costing several times that. An otherwise well-meaning IT employee (with no purchasing authority) could spin up a single $8/hour resource with no oversight, which by the time the bill has been received could total $6k-$7k in charges. While this situation would likely be recognized and addressed at the time the invoice was reviewed, dozens and dozens of smaller instances could take years of invoice cycles to clear up and over time have a much greater but less initially obvious impact. I have been involved in several remediation efforts for clients’ IT departments that were spending $500k+ annually in un-accounted for cloud services.
The most successful way I’ve seen enterprises address this new requirement is to create a cross-functional governance committee that includes representation from finance, IT and core business units, with the charter of managing cloud related costs.
Contrast this with the IT Provisioning Model where costs were governed prior to the time of purchase. When an IT department needed additional infrastructure, a Capital Expenditure process is/was in place that required finance approval. Budgets and expenditures were relatively easily managed, and without proper authority, individual purchasing power was limited. In the revolutionary world of 21stCentury IT, we need revolutionary methods of governance. The most successful way I’ve seen enterprises address this new requirement is to create a cross-functional governance committee that includes representation from finance, IT and core business units, with the charter of managing cloud related costs. In enterprises that have Cloud Steering or Governance Committees or a Cloud Center of Excellence, this cross functional group works under their direction. My good friends at Cloudability, who have developed what is probably the most comprehensive cloud cost reporting and management toolset in the industry, refer to this committee as the Cloud Financial Office (CFO – I believe the pun is intended). This committee evaluates the needs of the business, the reporting and cost management/containment requirements of finance, and the operational/support requirements of IT, to determine the best approach for meeting all three stakeholders. They develop strategy, policies and procedures for IT, finance, and business that lead to a deployed cloud infrastructure that is manageable from a cost perspective. As I mentioned above, there are tools that support this mission, but without the insights of the entire committee to interpret and act on the data, you will not recognize the value of the tools or succeed in being cost efficient with the capital you spend on cloud-based infrastructure. Tools are not a silver bullet. Just like there’s a nugget of truth underlying the health benefits of ketchup, when thoughtfully planned, considered and executed, you can recognize significant IT cost reductions as well as several other powerful benefits from transforming your infrastructure to the cloud. On the other hand, just like drinking a bottle of ketchup a day won’t cure or prevent any maladies, ignoring the revolutionary nature of the cloud and how your enterprise must adapt in order to “Cloud ConfidentlyTM”, won’t lead to any promised savings. It will likely result in higher costs for fewer benefits than you enjoy now. As you approach cloud adoption, remember, not everyone making claims of free and instant IT savings has your particular best interests at heart. Many of them, much like the 19thcentury ketchup hucksters, benefit handsomely as you overspend in the cloud. It’s the 21stCentury, don’t drink the ketchup, Cloud ConfidentlyTM!
Today, I want to expand on that concept and share one of the planning models that make cloud systems in particular and automated infrastructure in general more cost effective and efficient. When talking to clients I refer to this as “The Provisioning vs. Capacity Model”. First let’s look at the Provisioning Model, which, with some adaptation, has underpinned infrastructure decisions for the last five decades of IT planning. The basic formula is fairly complex, but looks something like this:
Let’s look at a practical example of what this means. As an IT leader, being asked to host a new application, I would work with the app vendor and/or developers to understand the compute, storage and networking configurations they recommend per process/user. Let’s say that we determine that a current processor core can support 10 concurrent users and a single user creates roughly 800K of data per day.
I would then work with the business to identify the number of users we expect to begin with, their estimate for peak concurrent users and what expected annual growth will be. Ultimately, we project that we will start with 20 users who may all be using the system at the same time. Within the first year, they anticipate scaling to 250 users, but only 25% of them will be expected to be using the system concurrently. By year five (our projected hardware lifespan) they are projecting to have 800 users, 300 of whom may be using the system at any given time. I can now calculate the hardware requirements of this application:
Being an experienced IT leader, I ‘know’ that these numbers are wrong, so I’m going to pad them. Since the storage is inconsequential in size (I’ll likely use some of my heavily over provisioned SAN), from here on out, I’ll focus on compute. The numbers tell me that I’ll need 2 servers each with 4 quad core processors for a total of 32 processors. Out of caution I would probably increase that to 3 servers. Configuring memory would follow a similar pattern. Because the application is mission critical it’ll be deployed in a Highly Available (HA) configuration, so I’ll need a total of six servers in case the there is a failure with the first three. This application will also require infrastructure in our DR site, so we’ll replicate those six servers there for a total order of twelve servers. In summary, on day one, this business would have a dozen servers in place to support 20 users.
The Provisioning Model can lead to overkill
Under the provisioning model a Highly Available solution with sufficient Disaster Recovery infrastructure could result in a large server deployment to support a very small number of users.
I know what you’re thinking, “This is insanity, if my IT people are doing this, they are robbing me blind!” No, they aren’t robbing you blind, they are following a “Provisioning” model of IT planning. The reason they plan this way is simple; it usually takes months from the time that an infrastructure need is identified to the time that it is deployed in production. It looks something like this in most enterprises:
1-2 weeks – Identify a need and validate requirements
1 week – Solicit quotes from 3 approved vendors (if the solution comes from a non-approved vendor, add 3 months to a year for vendor approval)
2-3 weeks – Generate a Capital Request with documented Business Justification
2 weeks – Submit Capital Request to Finance for approval
2-3 weeks – Request a PO from purchasing & submit to vendor
2-3 weeks – Wait for vendor to deliver hardware & Corporate receiving to move equipment to configuration lab
The total turnaround time here is 15-20 weeks. Based on the cost, time, pain and labor it takes to provision new infrastructure, we want to do it right and be prepared for the future, and there is no quick fix if we aren’t. Using a provisioning model, the ultimate cost in deploying a solution is not in the hardware being deployed, but rather in the process of deploying it.
The upshot of all this is; Most of your IT infrastructure is sitting idle or nearly idle most if not all of the time. As we assess infrastructure, it is not uncommon for us to see utilization numbers below 10%. Over the past 15 years as configuration management, CI/CD, virtualization and containerization technologies have been adopted by IT, the math above has changed, but because those technologies are evolutionary in nature, the planning process hasn’t. In the Provisioning model, we are always planning for and paying for capacity that we will need in the future, not what we need today. Enter Cloud Computing, Infrastructure Automation, Infrastructure as Code (IaC) and AI. Combined, these technologies have ushered in a revolutionary way to plan for IT needs. IaaS and PaaS platforms provide nearly limitless compute and storage capability with few geographic limitations. Infrastructure Automation & IaC allow us to securely and flawlessly deploy massive server farms in minutes. AI and Machine Learning can be leveraged to autonomously monitor utilization patterns, identify trends and predictively trigger scaling activities to ensure sufficient compute power is delivered “Just in Time” to meet demand, then scaled back as demand wanes. In cases where IaaS and PaaS providers experience localized outages, the same combination of IaC and AI can deploy your infrastructure in an unaffected region, likely before most of your user base or IT is even aware that an outage has occurred. Software updates and patches can be deployed without requiring system outages. The possibilities and opportunities are truly mind boggling. Taking advantage of these capabilities requires a complete change in the way our IT teams think about planning and supporting the applications our users consume. As I mentioned above, the incremental hardware costs of over-provisioning in the data center is inconsequential when compared with the often un accounted for cost of deploying that hardware. In forward looking IT, where IaaS and PaaS are billed monthly and provided on a cost per deployed capacity model, and infrastructure can be nearly instantly deployed, we need to abandon the Provisioning Model and adopt the Capacity Model. Before I proceed, you need to understand that these three pillars; IaaS/PaaS, Infrastructure Automation, and AI must all be in place to effectively take advantage of the cost savings and efficiency of the Capacity Model while still delivering secure, reliable services to your users. Merely moving (often referred to as “Lift and Shift”) your servers to the cloud and optimizing them for utilization may provide some initial cost savings, but at significant risk to security, availability and reliability of services.
3 Pillars of the Capacity Model
IaaS/PaaS, Infrastructure Automation, and AI must all be in place to effectively take advantage of the cost savings and efficiency of the Capacity Model.
Following the Capacity Planning model, we try to align deployed infrastructure to utilization requirements as closely as we can, hour by hour. You may have noticed in my Provisioning example above I was primarily concerned and planning for the required capacity at the end of the lifespan of the infrastructure supporting the application. I was also building to a standard that no system would ever exceed 35%-40% utilization. In the new capacity planning model, I want every one of my services running at as close to 90% utilization as possible. Ideally with only enough headroom to support increase in utilization for as long as it takes to spin up a new resource (typically only a few minutes). As demand wanes, I want to be able to intelligently terminate services as they become idle. I use the word “Intelligently” here for a reason; it’s important to understand that many of these resources are billed by the hour, so if I automatically spin up and terminate a resource in 15 minutes, I am billed for a full hour – If I do it 3 times in a single hour, I’m billed for 3 hours. Let’s look at a sample cost differential between Provisioning and Capacity modelling in the cloud. For this exercise, I’m just using the standard rack rates for AWS infrastructure. I am not applying any of the discounting mechanisms that are available and using simple calculations to illustrate the point.
It’s also important to remember that Revolution is neither free or easy; developing and refining the technologies to support this potential savings for this new application will cost $50k-$100k over the five years.
Provisioning Model – 5 Year Costs:
Total Cost: $352,512.00
Capacity Model – 5 Year Costs:
Total Cost: $14,551.32
In the model above, for simplicity of understanding, I only adjusted the compute requirements on a yearly basis, in reality with the ability to dynamically adjust both instance size and quantity hourly based on demand, actual spend would likely be closer to $8k over 5 years. It’s also important to remember that Revolution is neither free or easy; developing and refining the technologies to support this potential savings for this new application will cost $50k-$100k over the five years depending on the application requirements. At the end of the day, or at the end of five years, following the capacity model may result in spending well less than half the cost of the provisioning model, but you would have enjoyed much higher security, reliability and availability of applications with a significantly lower support cost. To wrap up this very long post, yes, it is true that massive cost savings can be realized through 21stCentury IT Transformation, but it will require a Revolution in the way you think about supporting your business applications. Without people experienced in these very new technologies, you’re not likely to be happy with the outcome. Finally, if you encounter anyone who leads the charge to cloud with words like “Lift and Shift”, please don’t be hesitant to laugh in their face. If you don’t you may end up spending $350,000+ for what could otherwise cost you $8,000.
I recently spoke with a good friend of mine who is a Finance SVP with a publicly traded North American manufacturer.
He was very excited to tell me that his executive team had been strategizing over the past couple of quarters and was getting ready to publicly announce they would be moving all IT services to the cloud and would be 100% complete by Q4 2019. As our conversation progressed, I asked him why they had made this decision and he offered several reasons, some of which were more valid then others. Ultimately, with some prior knowledge of how large, diverse, and (in several significant areas) outdated their technology estate was, I asked what their IT teams thought of this initiative. I’m pretty sure my jaw actually dropped at his reply, “Outside of the CIO’s office, nobody knows yet.”
Its not just all or nothing
There are a whole host of issues with the direction this executive team was headed, but for this particular blog I want to focus on one particularly poor decision that I see play out over and over with potential clients; they are either All In, or All Out on cloud adoption.
I would argue that the vast majority who take either of those two positions have a fundamental mis-understanding of what “Cloud” is. Since the term “Cloud” has been co-opted by nearly every vendor to mean almost anything, this misunderstanding is not surprising. For the purpose of my blog today, I’ll be referencing the key components of cloud computing: Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS). At their core, each of these offerings support the delivery of business application features to users. At the end of the day, delivering business application features to their users as effectively and efficiently as possible is, and should be, a primary concern of every executive. Executives hire very smart and talented teams of Architects, Analysts, Product/Program/Project Managers, Engineers and Administrators to accomplish this delivery. As these teams work with their respective vendors to understand the nature of the various applications they must operate, then design, build and configure systems to support them. To support diverse applications, these teams need diverse tools; and Cloud (IaaS, PaaS & SaaS) is only one. As powerful as the cloud may be, it is not always ideal, or in some cases, even suitable for every situation or application.
All In, or All Out on cloud adoption?
I would argue that the vast majority who take either of those two positions have a fundamental mis-understanding of what “Cloud” is.
Imagine for a moment that you are hiring the best contractor in your area to build you a home. You’ve worked with them to determine the ideal design, select the desired finishes, and come up with a budget and timeline. At what point would you think it was in your best interest to dictate to this contractor, the best in the area, what tools they may or may not use to deliver your finished home? Wouldn’t it be better to let them use the best tool for each individual job that needs to be done? If this is true, why do we as executives think it is in our best interest to sit in board rooms and determine what tools our IT teams may or may not use, without understanding the nature of the applications they need to operate? The simple answer: it is not. Rather than limiting the tools that their teams can make use of, executives, as the primary visionaries and strategists of the enterprise should develop guidelines for their teams. These guidelines will help them identify appropriate tools and strategies, ultimately helping them align themselves with the overriding executive vision. In our practice, we break these guidelines into two primary sections; Outcome and Bias statements. Outcome statements generally speak to requirements related to availability, reliability, durability and usability of applications, while Bias statements are a list of prioritized preferences for how the application is delivered. This construct provides for executive oversight, while also empowering teams to ultimately understand what is right and do it.
There are many enterprises that have gone all in on cloud adoption and many who have avoided it all together. In all my experience, I have yet to encounter an enterprise that has gone All In on the cloud without making significant compromises or undergoing supernatural gymnastics to get everything in (except for businesses that were born in the cloud). Likewise, I have yet to work with a business who has completely opted out of cloud that couldn’t benefit from having some of their systems residing there. “Outside of the CIO’s office, nobody knows yet.” As you can imagine, this jaw dropping statement was not the end of our conversation. We discussed the nature of Cloud Services, and he invited me to consult a bit with several of his peers and superiors within the organization. I was able to provide a little of my perspective and insight into the path they were preparing to undertake. The verdict is still out on what their plan for cloud adoption will be, but I have not seen them make a public announcement regarding a plan for a 100% cloud adoption by EOY 2019.
1st Century IT is a revolution – not an evolution of what we have been doing for decades. The skills required to transform through this revolution are different than those required to operate the existing state, which are different still from those required to operate the new state.
A fundamental misunderstanding of this concept underlies almost every troubled or failing IT transformation project. It took me a few years of assisting enterprises in their cloud migrations to fully understand the ramifications of the difference between evolution and revolution as it applies to IT initiatives and how they impact business.
First, let’s consider an earlier technological revolution
The combined impact of the Personal Computer, GUI and desktop publishing was revolutionary and had a transformative impact on the enterprise. Prior to this revolution, enterprises had specially trained computer operators to handle I/O functions for the mainframes and steno pools with typewriters for document creation. As a result of this revolution, there was a fundamental change to the way business was done. Employees, managers and executives alike were able, and quickly required to generate their own documents, manage their own calendars and perform their own data I/O. Within a short time, everything changed. Successfully navigating this change required different IT staff with completely different skills. It ushered in and set the tone for the next 3-4 decades of IT practices.
Within a short time, everything changed
Successfully navigating this change required different IT staff with completely different skills. It ushered in and set the tone for the next 3-4 decades of IT practices.
It’s interesting to contrast this with the evolution of virtualization that really took hold at the turn of the century. While virtualization had significant impact on IT Departments and how compute power was provisioned in the data center, it did not significantly change what IT staffs did, or how they did it. The skill sets required after a move to virtualization were mostly the same as those required prior to the move and the virtualization of data centers was primarily performed by existing IT staff. The impacts of this transformation effort were barely felt, if recognized at all, by those outside of IT.
I’ve spoken with countless enterprise leaders who view the transformation to 21st Century IT as nothing more than a data center migration – something that is normal to the ongoing operations of an IT department. While this can technically work, it’s unlikely to provide the ideal outcomes promised and sought after. Ultimately the reality will be that an IT estate moved in this manner will most probably cost more over the long term, negatively impacting security, availability and performance. The great news is that it doesn’t have to be this way!
If you embrace this transformation as a revolution, impacting all aspects of how your enterprise does business, you’ll be taking the first important step.
A few years ago, my team and I were brought into a large financial services company. They were looking to contain IT costs and as a result, investigating the cloud as a way to accomplish that goal – but this is not the start of the story. Over the previous several decades, this enterprise had become one of a couple “800lb Gorillas” in their particular vertical. They had thousands of employees, all the major customers, massive amounts of data and an annual IT spend nearing 9 digits. A few years prior to our involvement, a couple of start-up companies with few staff, no customers, no data and extremely limited IT budgets entered their vertical and started disrupting it. Initially, these start-ups were ignored by the enterprise, then they were mocked, and ultimately, as they began to take market share away, they were feared. The enterprise started playing defense, leading to the cost cutting exercise we were brought in to assist with.
As we worked with this client and helped them understand the revolutionary nature of the cloud and the wide-ranging impacts that it could have on the way they do business, they began to reevaluate their posture with regards to the insurgent companies. With the transformation to new technologies came a culture change. These changes positively impacted customer interactions and the speed with which our client was able to respond to feature requests. Eventually, our “800lb Gorilla” client became the disruptive innovator in their vertical. Today, aside from the name on the building and the vertical they serve, they don’t look much like they did when we first met them. The way that they do business has fundamentally changed; across their entire enterprise.
Your enterprise may or may not face similar challenges and you may not need or want such sweeping change, but regardless, understanding that your transformation is revolutionary, not evolutionary will position you well for success. Don’t be surprised if embracing the revolution doesn’t help address some of the issues you are facing.
Just be aware that it isn’t easy or free — revolution never is.
Digital Transformation Journey of a Global Restaurant Chain
A year long cloud transformation from a global network of physical data centers used to run company systems and applications.
One of the world’s largest restaurant chains offers technology services to its stores and franchisees that perform functions as diverse as POS systems, store management systems, data analysis and predictive analytics, digital advertisement and e-commerce and customer engagement platforms.
Their IT organization uses a small core staff with limited resources supported by a network of service providers and partners to fulfill key IT roles. Traditionally, the company used a global network of physical data centers to run their applications.
The company engaged with members of our team on key transformation initiatives
Need to build environments faster and with more flexibility than what traditional infrastructure in a data center could provide
Need to decouple scale up / out decisions from development cycles to speed the development process (get new apps and update apps faster)
Address local needs while maintaining global standards
Reduce CapEx to increase financial flexibility and agility
Lack of internal expertise in implementing and managing a global, enterprise public Cloud environment
Need to tie costs to specific internal projects and teams while benefiting from economies of scale
Following the completion of this engagement a prominent financial analyst commented: “The company is establishing a first-mover advantage with digital that can drive sustainable share gains in late 2017 and beyond.”
Reduced time to deploy infrastructure from 60- 120 days to minutes
Implemented Cloud automation for some workloads to enable same day deployments
Enabled global deployment of resources in AWS regions close to end customers / users to increase performance and decrease latency
20-50% cost reduction on next generation deployments for customer engagement and data warehouse projects vs. traditional models
Enabled internal bill back of resources in AWS to specific projects and teams
Following the completion of this engagement a prominent financial analyst commented: “company is establishing a first-mover advantage with digital that can drive sustainable share gains in late 2017 and beyond.”
A longtime leader in golf equipment and apparel needed to find an alternative disaster recovery solution for a new ERP system.
Finding itself with a number of legacy IT systems, the company was looking to upgrade their infrastructure in a number of areas. With an upcoming planned elimination of a corporate disaster recovery support platform, IT management saw an opportunity to investigate alternative solutions for their DR requirements.
“We wanted to upgrade our disaster-recovery capabilities in order to mitigate the chance of data loss in our mission-critical, enterprise resource planning or ERP system,” said the Director of Infrastructure and Services.
“We were looking at the concept of continuous data protection in both our onsite production and DR environments,” he added. The company also wanted to incorporate newer technology, which would allow for quickly scaling memory size, CPU and disk space – without having to purchase incremental hardware.
While they were using nightly backup and data replication for disaster recovery, the company envisioned a solution with a lower recovery point objective (RPO) through continuous replication. They required a best-of-breed disaster recovery environment to match the 99.99 percent uptime of their new Oracle ERP solution.
The planned elimination of a legacy DR platform provided an opportunity to modernize.
“From the very beginning, we were talking about instances and hourly costs. This was an entirely diﬀerent approach from the colocation options we explored earlier.”
To achieve their vision of a scalable DR environment, the company needed to look beyond colocation. Our experts helped the company focus on finding a suitable DR as a Service and cloud solution.
“For a long time, we did not think our requirements would work with Amazon. We required private networking and multiple nodes to be replicated synchronously, that seemed to defy implementation at a public cloud provider,” said the director of infrastructure and services.
To facilitate disaster recovery of its ERP database, the company decided on an Oracle Limited disaster recovery optimized solution. “We learned that the Oracle Limited solution was available to us at no cost when in sleep or standby state.”
AWS is an authorized cloud platform for Oracle — one of a very small number of approved cloud vendors.
It was the flexibility and willingness to share its operations expertise that attracted the company to effectual’s team. “From the very beginning, we were talking about instances and hourly costs. This was an entirely diﬀerent approach from the colocation options we explored earlier.”
An economical cloud-based, disaster-recovery environment oﬀering the potential to do more with less.
The company deployed the architecture for its disaster recovery platform on Amazon Web Services. “We can even move between various Amazon data centers if needed for changing protection requirements – without incurring any data transfer charges.”
The effectual team was able to build a custom, and cost-eﬀective, DR environment harnessing the power of AWS. The company had a highly specific use case for the deployment of cloud resources for disaster recovery in an AWS environment. It was an ideal opportunity for Effectual to architect and secure an optimized solution at scale.
How to Tell Ahead of Time If Your IT Transformation Project is Going to Fail (1 of 7)
We’ve started using a phrase in my office that encompasses all of the various migrations, reorganizations and modernization trends going on in IT departments today: “21st century IT transformation.”
Yes, I know. It’s a mouthful. Someone smarter than me will probably coin something more concise. And catchier. The phrase refers to the adoption and combination of technologies such as “Continuous Integration/Continuous Delivery” (CI/CD), “Infrastructure as Code” and “Artificial Intelligence” (AI); and methodologies such as “Agile” and “DevOps;” and service models such as “Cloud Hosting.” Combined, these areas help IT organizations meet business requirements and deliver business value.
Learning from Failures
I’ve been working in the IT Ops space for more than 30 years. The last 10 of these years have been spent assisting clients of all sizes understand and ultimately make the transformation to 21st century IT models and approaches. While I have a great winning percentage overall, I am experienced enough that my scorecard also reveals a few failed projects over the decade; that is, efforts that produced neither the desired business outcomes nor technical outcomes.
Success requires the support of key leadership
There is a fundamental misunderstanding at the executive level of the potential impacts of new concepts such as DevOps, and CI/CD, and principles such as “Fail fast, fix fast.”
During my journey, I’ve been able to learn from my own mistakes as well as from those of others – I’ve observed many projects without being involved. And, while every project is unique in its own way, I’ve recognized that there are a few hallmarks that almost always foretell failure. In this week’s blog, I’ll highlight several of the most prevalent warning signs from a high level. Then, in each subsequent post over the coming months, I’ll go increasingly into more and more depth. I’ll write primarily from a business perspective in this series because most transformation projects that will ultimately fail can be identified before the first engineer is assigned. This comes from a fundamental misunderstanding – at the executive level – of the potential impacts of new concepts such as Cloud Adoption, DevOps, and CI/CD, and principles such as “Fail fast, fix fast.” A quick note before I go into this week’s list. Your particular cloud initiative is not necessarily doomed to failure because one or two of the factors described below apply. That being said, the odds can quickly tip in favor of failure as more and more applicable issues appear.
The List of Dreaded Pitfalls
1. You did not start your transformation with an agnostic application or business feature-based assessment of your current IT estate.
This, more so than any other item in this list, will – if not properly performed – lead to budget/timeline overruns, failed deployments and ultimately unhappy internal/external clients. A properly performed assessment should answer the following questions, at a minimum:
What are the business requirements of my IT estate?
Why should I transform my IT/What do I want out of it?
What is my current application/business feature inventory?
For each business feature, what is (or are) the:
Actual infrastructure requirements
Currently deployed infrastructure
Cost of operation per month
Business Continuity/Disaster Recovery posture
Actual cost (in lost productivity/revenue) of availability per hour and workday
Associated development, Quality Assurance, Configuration and sandbox environments
Based on the inventory above, what functionality or performance-related issues need to be proven out through Proof of Concept efforts before ultimate decision are made?
What is the appropriate high-level budget/timeline required to complete this work? It’s easy to get lulled into the complacent thought that you’ve been operating your infrastructure and applications for a long time, so you know it very well. In practice however, the knowledge required to operate is not the same as the knowledge required to transform.
This leads to the next red flag:
2. You consider the transformation effort to be an evolution of your previous IT models, practices and tools.
21st century IT is a revolution – not an evolution of what we have been doing for decades. The skills required to transform through this revolution are different than those required to operate the existing state, which are different still from those required to operate the new state. Many efforts have failed because the executives responsible did not understand that a cloud migration does not and should not resemble a data center migration – regardless of what the various migration tool and cloud hosting partners will tell you.
3. Without first performing an assessment to evaluate the skills or effort required, you and your senior IT staff dictated the timeline, budget and technology decisions.
This seems so absolutely ridiculous that it can’t possibly be true. Could you imagine going to a heart surgeon and demanding a transplant, without understanding the impacts or even if the transplant was needed? Of course not. But somehow enterprises do this every day with the beating heart of their businesses – a.k.a. IT – without giving it a second thought.
4. Prior to starting your transformation, you didn’t have a complete understanding of the financial and operational models behind 21st century IT in the enterprise.
Yes, you understand OpEx vs CapEx, and are maybe even able to make a solid “Net Present Value of Cash” argument regarding your future IT directions. But have you considered Provisioning vs. Capacity planning models? Do you understand the cost and value related to Infrastructure as Code? Can you articulate the risks inherent in Best Efforts Availability as opposed to Five Nines? How will you manage costs in a world where a single button click can result in thousands of dollars in Monthly Recurring Costs?
5. You view the transformation as a strictly technical effort.
This is a big one. To make IT more efficient through transformation, you have to address two areas: The How and the What. Assigning the work to your IT organization addresses the How. Involving business resources drives the definition of the What. Without both groups working together, operational efficiencies won’t be realized, budgets will be blown, and opportunities lost. If some of the issues listed above are factors in your current transformation effort, it’s never too late to try to resolve them. In the coming months, I’ll expand on the thoughts above and share some anonymous war stories while providing pointers on how to avoid pitfalls in the business of 21st century IT transformation.
21st century IT is a revolution – not an evolution
The skills required to transform through this revolution are different than those required to operate the existing state, which are different still from those required to operate the new state.
Transforming an existing legacy environment and building it natively on AWS.
A leading social media company’s apps explore the humanness of people instead of simply quantifying how good they are, or judging how professional their work may be. Users can upload their work directly to their platform, using it to edit photos, with enhancements – including white-balancing, filters and journaling – readily available. One user documented his mother’s fight with cancer, displaying the emotions he and his family felt during a very trying time.
In 2015, the company acquired a platform that creates tangible photo books, prints and gifts for digital photos. The platform was using a non-native environment and didn’t believe it was the right strategic technology in the long term. The combined companies have more than 30 million monthly active users across its platform consuming 5 billion images. Reliable infrastructure users can depend on is essential. The company wanted to work with experts who to help them transform their existing legacy environment and build it cloud natively on AWS.
30 million monthly active users consuming 5 billion images
Reliable infrastructure users can depend on is essential. The company wanted to work with experts who to help them transform their existing non-native environment and build it cloud natively on AWS.
The companies DevOps team is very capable – they had significant experience with Chef, using it to configure their servers and developer machines – but the transformation to AWS required resources the team just didn’t have.
The transformation to AWS required resources the company’s team just didn’t have.
30 million monthly active users consuming 5 billion images.
Reliable infrastructure users can depend on is essential. The company wanted to work with experts who to help them transform their existing non-native environment and build it cloud natively on AWS.
Since migrating to AWS, the company has seen great benefits: “Operationally, AWS is faster and more configurable than what was being used,” said the VP of Engineering at the company. “The new environment is faster, more reliable, and cost efficient. All three are pretty important things!”
Most importantly, this solution had a big impact on the stability of the company’s platform and by extension, it’s brand. Previously, the platform had experienced random reboots on several occasions. The successful transition to AWS allowed them to have confidence going into the holiday season, their most important quarter of the year. Confidence in the reliability and performance of their platform allowed the company to focus on maintaining a great experience for their user community.
“The experts had both the attitude and the aptitude, and I feel as comfortable as I possibly could in having AWS as the long-term foundation for our infrastructure.”
Bring in the Experts
With the support of our experts the company was able to seamlessly migrate from its existing environment to AWS. From the start, the company was impressed with our experts knowledge and expertise. “They inspired a lot of confidence, and their team clearly had the technological expertise,” said the VP of Engineering. “We knew they could get the job done.”
Our experts leveraged their extensive experience with Vagrant and Packer to build a strong foundation for the company’s cloud native environment. This enabled the company to emulate the user experience of their members. The insights they gained helped them improve their product, streamline their operations and deliver an optimized user experience.
“Technology expertise is the number one thing we look for when hiring an outside firm,” said the VP of Engineering. “The experts had both the attitude and the aptitude, and I feel as comfortable as I possibly could in having AWS as the long-term foundation for our infrastructure.”