Leveraging Amazon EC2 F1 Instances for Development and Red Teaming in DARPA’s First-Ever Bug Bounty Program
This past year, Effectual’s Modernization Engineers partnered with specialized R&D firm Galois to support the launch of DARPA’s first public bug bounty program – Finding Exploits to Thwart Tampering (FETT). The project represents a highly unique use case showcasing Effectual’s application expertise, and was approved this week to be featured on the AWS Partner Network (APN) Blog.
Authored by Effectual Cloud Architect Kurt Hopfer, the blog will reach both AWS customers and technologists interested in learning how to solve complex technical challenges and accelerate innovation using AWS services.
In 2017, the Defense Advanced Research Projects Agency (DARPA) engaged research and development firm Galois Galois to lead the BESSPIN project (Balancing Evaluation of System Security Properties with Industrial Needs) as part of its System Security Integrated through Hardware and Firmware (SSITH) program.
The objective was to develop tools and techniques to measure the effectiveness of SSITH hardware security architectures, as well as to establish a set of “baseline” Government Furnished Equipment (GFE) systems-on-chip (SoCs) without hardware security enhancements.
While Galois’s initial work on BESSPIN was carried out entirely using on-premises FPGA resources, the pain points of scaling out to a secure, widely-available bug bounty program soon emerged.
It was clear that researchers needed to be able to stress test SSITH hardware platforms without having to acquire their own dedicated hardware and infrastructure. Galois leveraged Amazon EC2 F1 instances to scale infrastructure, increase efficiencies, and accelerate FPGA development.
SpiraLinks: Rapid migration to AWS unlocks new cloud-native capabilities
SpiraLinks offers tailored consulting services for projects, technical event, and implementation management to Fortune 500 companies, including designing, installing, and hosting secure web-based systems for human resources, compensation, and finance teams. The company’s FocalReview® planning suite is a leader in compensation and performance management, supporting customers in the US and beyond.
Driven primarily by the upcoming consolidation and closure of the data center hosting its product platform, SpiraLinks had made the strategic decision to migrate its infrastructure to the AWS Cloud. This included three application servers, a legacy Oracle database environment, and an older standalone Windows application. The company also had several virtual machines that were being retired by their MSP.
SpiraLinks recognized that a successful migration would provide an opportunity to modernize its technology stack and leverage new AWS capabilities to better serve its customers. However, without the internal resources to accomplish the move, the company needed to engage a partner with the technical resources and expertise to achieve the migration.
Benefits of the AWS Cloud
The company chose to migrate to an AWS environment to increase efficiencies, improve security and compliance, and optimize costs. In addition, the SpiraLinks wanted to access new AWS Native services to modernize and evolve its business.
Outsourcing Migration Expertise to Effectual
To achieve these business objectives, SpiraLinks partnered with Effectual to lead its migration and modernization efforts. Effectual is a cloud first, security first managed and professional services company and AWS Premier Consulting Partner with deep expertise leading complex migrations and managing modern cloud environments across VMware, VMware Cloud on AWS, and native AWS environments.
Solutions & Outcomes
Completed a full migration of customer-facing applications from on-prem infrastructure to a new, modern, secure AWS environment in less than a month.
Deployed all new modern Linux and Windows servers in separate VPCs for improved security
Configured Amazon Elastic Block Store (EBS)for the three Linux EC2 instances hosting Wildfly (formerly JBoss)
For Oracle database server:
Migrated all data from legacy Oracle environment
Upgraded and deployed database into a new Amazon Relational Database Service (RDS), allowing for adoption of Session Manager for accessing application servers (improving security and decreasing costs) and providing added functionality with real-time performance insights
Increased the security layout of the data environment by isolating it in its own private subnet and restricting access
Restricted access via approved ports from application servers
Deployed to a single AWS RDS instance with individual database schemas
Replicated and enhanced mail sending capabilities to utilize Fluent Ltd. mail relay service.
Increased security due to inherited ISO certification from AWS.
Created an AWS Identity and Access Management (IAM)group and defined the IAM policy to provide SpiraLinks developers with access to the AWS Systems Manager Agent(SSM). Once the IAM groups and policies were configured, shared initial login credentials with the primary SpiraLinks contact and configured Multi-Factor Authentication (MFA) to enhance solution security.
Deployed and configured CloudTrail and CloudWatch EC2 log streams to monitor instances, and also configured email alerting for these services.
Configured Amazon Data Lifecyle Manager (Lifecycle) to take snapshots, with a rolling 14-day retention period.
Established a clearer understanding of data needs as well as the specific benefits of AWS environment and services in order to make informed choices.
Through its partnership with Effectual, SpiraLinks was able to achieve a rapid migration of its infrastructure to the AWS Cloud and avoid unexpected downtimes associated with the closure of its MSP’s data center. The migration to an AWS environment provided opportunities to improve security, increase efficiencies, and optimize costs while opening new pathways to modernizing using AWS native services and capabilities.
Moving forward, SpiraLinks will utilize the newer, more secure AWS environment for its many tools and benefits in accordance with the constantly changing business and operational requirements of the SpiraLinks client base. Specifically, compliance and data protection/privacy will be evolving challenges for SpiraLinks and the client base. The AWS environment has been chosen as an excellent “base of operations” to meet those challenges.
SpiraLinks will continue to work with Effectual as a Modernization Service Provider to utilize their expertise in addressing the company’s long-term goals and challenges. In addition, SpiraLinks and Effectual have developed an evolving roadmap that includes further modernization efforts to increase automation, availability, reliability, and security – further establishing the position of SpiraLinks as an industry leader.
About Time Tours: Guiding a Successful SaaS Journey
About Time Tours is a Pacific Northwest startup redefining how the real estate industry plans, organizes, and coordinates home tours between agents and homebuyers. With market expertise but only a general business idea, the company asked us for help developing a SaaS-based solution. We guided them on their SaaS journey from a basic app concept to a scalable production-ready launch using AWS SaaS services and best practices.
As a startup, About Time had already identified key pain points facing both realtors and home buyers for scheduling home tours. For all involved, the existing process was time consuming, cumbersome, and fraught with unnecessary complexity. About Time saw an opportunity to streamline scheduling and communication and capture feedback. The company also wanted to maximize the market opportunity and go to market as quickly as possible.
Building the business case & defining the product vision
Given almost 50% of our professional service engagements are SaaS focused, we have deep experience implementing the SaaS business model for clients. We started with About Time by embarking on a full discovery process, beginning with building a well-defined business case for their solution and outlining their product strategy.
This included evaluating customer pain points, developing user stories, and creating a seamless UX/UI experience. We also conducted a competitive analysis of off-the-shelf solutions to determine what problems they solved, how they solved them, and their challenges. After establishing the business case and product vision, we built a series of wire frames showing functionality of features and workflow before moving on to mockups of the app.
Aligning the MVS with AWS SaaS best practices
For SaaS clients, defining a Minimum Viable Service (MVS) always poses the greatest challenge. It is also the most critical stage on the SaaS journey where resources are concerned, as you can easily over architect your solution and run up costs. We worked with About Time to decide on the right MVS, knowing they would receive important feedback after going live that would likely change the app in future sprints.
Once we had defined the MVS, the AWS SaaS Enablement Framework provided a clear, thorough process for us to evaluate tenancy, security compliances, and compare cost models against the company’s revenue objectives. We also helped About Time prepare documentation and collateral in support of their efforts to secure investor funding.
From development to launch -Leveraging the Well-Architected Framework
In the next phase, our development efforts followed an agile process with milestones sprint by sprint and continual, transparent communication with About Time’s founders and investors. We used the Well-Architected Framework to ensure we were properly evaluating tradeoffs and applying cost optimization strategies when it came to reliability and security. We also segregated their personally identifiable information (PII) data in a multi-tenant environment to meet security compliances.
In addition, we built their app 100% on serverless so it can scale rapidly as their user traffic increases and utilized a pay as you go model to keep costs per user in line with their profit margin and revenue expectations for sustainability and growth.
For testing, our team conducted performance tests to ensure the app could handle expected traffic and security tests to confirm there were no hacking activities. We also held an informal gameday so that there is support documentation in place in case the app goes down during a live environment. Last, our SLA with the company sets expectations regarding our response time and steps we will take to ensure they are up and running quickly.
At launch, About Time’s final mobile and web app represents a highly scalable SaaS solution capable of growing with market demand without compromising on security and cost. Our next steps include capturing feedback and optimizing features and workflows to keep customers happy and satisfied with their solution.
Working with the Effectual team to refine our MVS gave us an objective view of how to align our revenue goals with the right cost model so we could take an informed approach to choosing the best SaaS strategy. As we scale, we know we can meet our business objectives and deliver a high-quality customer experience.
Tricon American Homes: IT modernization helps industry leader keep pace with accelerated growth
Tricon American Homes (TAH) is the fourth largest single-family rental company in the US. Founded in 2012, the publicly owned company has a portfolio of over 20,000 homes in ten states. As one of the country’s fastest-growing real estate companies, Tricon has gained a competitive advantage by offering highly responsive, personalized customer service and translating it into profitable long-term relationships.
Since 2016, the company has more than doubled its rental home portfolio. This accelerated growth highlighted the need to formalize and streamline processes, reduce costs, and optimize its operational efficiencies. With properties spread out over large geographic areas, Tricon was also in search of scalable solutions for managing, servicing, and maintaining their homes as well as a responsive communication platform for delivering a high-touch, seamless experience to its residents.
Challenges to pursuing these strategies included limited internal development resources as well as a lack of off-the-shelf solutions for the single-family vertical market. As the company expanded, Tricon partnered with a specialized team of solutions architects at Effectual to integrate their business requirements with DevOps expertise and take advantage of evolving Amazon Web Services (AWS) solutions.
Improving operational efficiencies while scaling teams and services quickly
Combining multiple data sources to create complete and holistic reporting
Innovating continuously to optimize costs and meet market demand
Creating a DevOps culture focused on automation, cross department communication and collaboration
During the last four years, Effectual has supported these goals by designing, developing, and deploying numerous solutions for the company leveraging the AWS Cloud. These include applying IoT capabilities, integrating smart home technologies, and utilizing AI/ML managed services for revenue enhancements.
Solutions & Benefits
Operational Efficiencies One of Effectual’s first projects was to streamline Tricon’s existing rental process and leverage automation to integrate existing administrative functions with custom business applications. In addition, the team developed and launched a 200-home smart home pilot with BeHome247, a cost-saving program Tricon is rolling out to its entire portfolio.
Continuous Integration – Continuous Deployment (CI/CD) To speed time to market deployments, increase reliable releases, and provide a secure environment, the team also created a CI/CD stack and pipeline that aligned new feature requests from ideation to deployment.
Performance & Functionality In order to reduce AWS costs and increase scalability, Effectual streamlined Tricon’s application payments, built a performant dashboard, and deployed AWS S3 for highly scalable cloud object storage.
Monitoring & Logging Last, by integrating all aspects of custom applications with CloudWatch, Tricon can now easily monitor and quickly troubleshoot issues without affecting their customer experience.
The success of these initial improvements has led Tricon to further expand its partnership with Effectual, including exploring new AWS services and developing additional custom applications to better serve its residents. The company is rapidly become an industry leader in new technologies.
“Effectual has been an extension of our team for several years, and we appreciate their focus on implementing scalable, innovative, and well-architected solutions in partnership with us. They continually go above and beyond to ensure that new projects are successful by asking questions and clarifying assumptions to truly understand our business objectives. We utilize their knowledge to evaluate new technologies and services to ensure that our technology stack is optimized.”
– Dawn Dalton, VP of Business Systems
Implementing Modern Cloud Management & Optimization
As the team addressed Tricon’s development requirements, it recognized the company was going to need ongoing management to monitor and maintain the security and performance of its AWS platform as well as to identify opportunities for cost optimization and business intelligence strategies.
With a growing estate of applications running on AWS continued, it became apparent that opportunities existed to improve the maintenance and security of Tricon’s environments. As a cloud-first, security-first Modernization Service Provider, Effectual provided the experience and expertise to keep the company on a path of continued innovation.
In particular, Tricon had experienced disruptions within critical business systems and wanted to improve their response time with greater visibility into what was causing errors. By establishing automated monitoring alert systems, Effectual has helped Tricon to respond quickly and resolve issues as they occurred, reducing downtime and improving customer experience.
“We’re coming to resolution much faster now on issues. Before working with Effectual, it was taking us longer to figure out the root cause.”
– Gregg Knutson, Sr. VP of Information Technology
With consistent reporting, Effectual’s delivery team has also been able to uncover patterns affecting Tricon’s costs. During a recent quarterly business review, Effectual reviewed the company’s 6 month cost trends and identified an unutilized RI (Reserve Instance) that had become orphaned. Resolving the issue helped Tricon take advantage of the significant discount RIs offer – a key strategy for cost optimization. This proactivity is one of the most important benefits of having a long-term partnership with a Modernization Service Provider.
As a trusted advisor, Effectual’s overall goal is to set Tricon on a path of scalability and growth with the confidence it can securely and reliably serve its customers. This includes aligning the company’s business goals with new tools, methodologies, and strategies to support their growing business.
“We try to be a forward-thinking organization in terms of technology, and really want to leverage modern IT systems. With this partnership, we’ve deepened our ability to meet the growing needs of our technology roadmap.”
Xenon, Inc: IoT Proof of Concept Accelerates New Market Opportunities
Xenon, Inc. is a custom hardware provider offering full-service engineering, integration, and field service solutions for the oil and energy industries. The company provides process and environmental analytics, industrial instrumentation and automation, and electrical systems.
Though Xenon primarily serves industrial markets, the firm was approached in 2018 by a new customer interested in applying their industrial engineering background to building IoT solutions for optimizing home maintenance, monitoring, and asset protection. An institutional single-family residence company with a portfolio of thousands of homes, the client was particularly interested in testing automated door locks, water sensors, and other smart devices for secure access and efficient maintenance. Their proposed plan included deploying devices in vacant properties each month with a three-year installation phase.
Partnering with Xenon provided an opportunity to explore and validate the impact of installing IoT smart home solutions for improved customer experience and reduced operational costs. For Xenon, the project presented a new market outside of its industrial focus. To respond, they needed a proof of concept to test in the first 200 homes and present to executives.
Leveraging expert advice for faster proof of concept
Xenon began building the IoT platform in Amazon Web Services (AWS) on its own, but soon encountered issues. As hardware engineers, they realized they were outside their core competency and needed help from experienced solutions architects on software integration with their client’s property management system. They engaged Effectual to review their existing architecture and implement Well-Architected best practices.
Xenon’s primary challenge was creating a cost-efficient cloud architecture that could scale. When the Effectual team conducted an initial review of the company’s environment, we confirmed the existing software layer would require fundamental changes to meet their cost requirements. In addition, our evaluation revealed the platform was built on one computer with no staging environment and no redundancy. This existing environment jeopardized the long-term reliability and scalability of the platform.
Based on this analysis, our team estimated Xenon would quickly out-capacity their existing environment at 100 homes. This was insufficient, as they needed to prove they could scale rapidly to service the company’s expanding property portfolio. Effectual also felt Xenon’s small development team could benefit from mentoring and guidance on key concepts and AWS IoT Core best practices.
Key recommendations and outcomes included:
Built a scalable, reliable proof of concept that met the client’s business requirements and budget
Confirmed AWS as the right solution for expanding their offering
Established DevOps best practices and trained internal team on processes
Educated company on costs and complexity of creating an IoT solution on a traditional infrastructure with EC2, load balancers. Showed them the significant benefits of using a serverless framework to process IoT events from Amazon Kinesis and device command management.
Developed 187 AWS Lambda functions for an estimated 40,000,000 events per month.
Implemented Amazon Kinesis to collect, process, and analyze 60,000 incoming records per day (30 MB of streaming data per day) to provide reliable, real-time insights and rapid response capabilities.
Deployed AWS API solution with an advanced logging and control layer for Xenon’s large scale IoT system to handle a high volume of burstable requests. Designed one gateway to ingest IoT alarms and events, and another to receive commands from external systems and applications.
Implemented Amazon DynamoDB as the primary storage mechanism for scalability with all tables using On-Demand for capacity control.
With Effectual’s help, Xenon responded quickly with a functional, reliable proof of concept that addressed their client’s pain points and met their business requirements. They validated AWS as the best cloud solution for propelling their project forward and gained a solid understanding of AWS IoT services.
Results & Next Steps
For their client, the project provided a better grasp of the costs and resources needed to deploy smart home systems in their properties. It also revealed what checks and balances they need to put in place for their operations.
From Effectual’s perspective, these outcomes are precisely what a successful proof of concept project should accomplish. If the client does decide to roll out these systems to its entire portfolio, we look forward to helping Xenon revisit its current configuration with some new approaches to further unlock the potential of the AWS Cloud.
Verdant Web Technologies: AWS tools improve scalability, profitability, and customer experience
Verdant offers management software solutions to track, access, and update facility Environmental Health & Safety (EH&S) compliance and sampling information.
With a growing customer base and a maturing product, Verdant was starting to encounter big DevOps and infrastructure challenges that threatened to slow its market momentum. The company’s migration of thought and concept demanded a far more scalable model. To the Effectual team, it was clear that the AWS platform could help them pivot and evolve.
Standardized architecture improves DevOps
Verdant’s primary pain point was architecture. With six different code bases unique to each client, the company updated code changes manually, published them out to 10+ web servers, and ran its SQL scripts on multiple databases. The process was overwhelming their team, impacting scalability, and preventing them from writing new features. The company’s IP also lived with a single developer, creating some vulnerability. Our team immediately got to work rewriting the company’s software with multi-tenant support, allowing different organizations to manage their data separately but with a standardized code base.
Streamlined DevOps by automating the deployment/development process with a build server and rapid deployment tools
Created a faster, more reliable migration to the AWS Cloud
Leveraged AWS for greater security and global redundancies to safeguard against potential downtimes
IP knowledge is now shared broadly by Verdant’s entire team so the company is no longer reliant on one person to protect its IP
The ability to scale rapidly to meet customer demand
“Effectual has been an amazing partner in the development of our enterprise platform which is now our life blood. Along with their responsiveness, solution engineering depth and capabilities we appreciate their tight management of project budgets and schedules. Effectual is a valued resource and critical part of the Verdant Team!”
– Ron Petti / President, Verdant Web Technologies
Eliminating hardware lowers cost of customer acquisition
Before deploying AWS, it took Verdant weeks to onboard new clients with a process that required significant hardware investments. Infrastructure was a fixed asset regardless of the number of clients. Our solutions turned infrastructure into an operating cost and eliminated hardware altogether.
Reduced new client onboarding from 2 weeks to 1 hour
Eliminated need for costly hardware
Decreased customer acquisition costs
Scalable solution allows for fast response to market demand
For Verdant, the timing for the project couldn’t have been better. Shortly after its completion, the company’s client base exploded overnight when schools around the US were compelled to perform extensive drinking water testing in reaction to the national crises in Flint Michigan. The revelation resulted in stricter reporting requirements and EH&S monitoring across Oregon, driving sudden intense demand for Verdant’s software solutions. With Effectual’s help, the company was well positioned to capitalize on incoming project opportunities, which resulted in a national award (with Environmental Business Journal)
AWS solutions such as Elastic Beanstalk support continuous development and innovation and help Effectual manage multiple application environments for the development/testing/release cycle
Increased customer satisfaction with ability to quickly add new functionalities
Tourvast: Building SaaS Solutions Using Scalable Innovation
Tourvast is a Software as a Service (SaaS) provider with a marketing platform that offers real estate photographers tools for creating property presentations and virtual tours that showcase their skills, leverage their art, and build their business. The platform also offers agents the opportunity to enhance their brand across social networks with high end, quality photography and video assets.
While the platform had been in existence for over a decade as licensed software, Tourvast executives wanted to evaluate the company’s intellectual property and consider options for writing their own application for greater usability. With new business requirements and a new go-to-market strategy, they contacted Effectual for help with their decision-making process and next steps. Our team provided insights and strategic advice and ultimately implemented a more scalable platform based on the secure, reliable infrastructure of the Well-Architected Framework.
Recalibrating the pricing model
In 2016, Effectual solutions architects began a discovery process that included wire-framing and architecture planning. This exposed one of Tourvast’s primary challenges, which was the inability to scale its pricing model. Due to the unpredictability of its customers’ large media files, the current architecture was not consistently covering costs.
Conducted an in-depth revenue modeling analysis to identify average costs based on number of photos uploaded as well as the number of videos, pdfs, and other assets created.
Designed new architecture for cost objectives with pay-for-use pricing to reduce capital expenses.
Improving performance, scaling for demand:
In addition, tenant activity was slowing performance and impacting overall customer satisfaction and retention.
The existing workflow began with a transaction outside of the platform between a real estate agent and photographer to secure photos for the creation of marketing deliverables. Photographers paid a subscription fee for a specific number of properties in advance and banked them like a credit system. After taking photos of the agent’s identified property, they uploaded their images on the website in order to organize them into deliverables such as slideshows, virtual tours, flyers, and more. Once complete, they provided their realtor customer links to those assets. Upon the agent’s approval of the copyrighted materials, they pay the photographer the invoiced amount through Tourvast to release the media for use.
The challenge was that each time a photographer uploaded their image files, the software would immediately resize them and create a slideshow. This process would take up the site up to 10 minutes while the photographer waited for it to complete. At the same time, it froze platform functionality for all customers on the site.
Leveraged serverless architecture using S3 and AWS Lambda for media and multi-tenant loads, resulting in greater flexibility and stability.
Implemented CloudFront for streaming videos to deliver content to end users with lower latency.
Deployed a blue-green architecture on AWS creating a continuous integration/continuous deployment (CI/CD) pipeline, including up to 10 servers for burstable traffic.
Code is now developed and deployed to an AWS Elastic Beanstalk environment, with two separate, but identical, environments (blue and green) to increase availability and reduce risk. This allows the application to continues to run seamlessly while new code is deployed without impacting the user experience.
Implemented DevOps strategies and best practices with parallel development, testing, staging, and production environments.
Ensured that no development takes place in production
Created a testing environment for internal QA
Enhanced reliability with a staging environment built for “friends and family” releases with a copy of production data scrubbed for security reasons (with scale of data to mimic what happens in production)
Today, Tourvast is a SaaS company that owns its own intellectual property, with full control over its roadmap. With support from Effectual, it owns its maintenance backlog and understands its third party dependencies and costs. Last, our team continues to help the company innovate and build improvements using proof of concepts fueled by cost-effective AWS tools.
Pole Pedal Paddle: Cost Optimization Strategies Maximize Impact for Local Non-Profit
Held in Bend, Oregon, the SELCO Pole Pedal Paddle (PPP) event is a popular annual multi-sport relay race benefiting Mt. Bachelor Sports Education Foundation (MSBEF). Over 2500 amateur and pro athletes compete solo or as a team in the race each year. The six legs of the race include Alpine skiing, Nordic skiing, biking, running, a canoe/kayak/SUP leg, and a sprint to the finish. Race Manager Molly Cogswell-Kelley asked Effectual for a custom solution to improve race registration, team management, and results reporting as well as to reduce PPP costs.
One of the main cost drivers was an expensive yearly subscription service with a third party provider. With two races (the adult PPP and the kid’s PPP), the event was paying for two subscriptions. In addition, even though their production only ran 4 ½ months each year, they were being charged for an entire year. The off-the-shelf service also lacked needed functionality for managing PPP’s different categories and unique team structures. In addition to growing costs, the organization had a new legal requirement for attaining parent signatures for child waivers.
Applying Well-Architected Framework best practices for cost optimization, our team worked on several strategies targeting the PPP production environment and development process to address business requirements and meet cost objectives.
First, it was clear that an annual subscription was a poor pricing model for the once a year event. We shifted to a more flexible, pay-as-you-go solution with AWS to ensure usage and costs match PPP’s short production timeline. Using AWS allowed PPP to run servers and pay for usage only when their environment was turned on. Given that our development team has nine months each year to develop new features and push out code, we also decided to use spot instances for the development and CI/CD environment, representing a 90% discount.
For the same amount PPP was paying each year to manage its waivers, we were able to build a custom application with far greater functionality and flexibility to accommodate ongoing development needs. Due to the nonprofit’s budget constraints, we focused on the adult PPP race as the first phase of the project. We were able to build on this framework two years later for the kids PPP.
To optimize costs, Effectual leveraged their existing knowledge with .Net membership provider, .Net user provider and the .Net MVC frameworks and libraries. This allowed Effectual to focus only on the custom business logic and leverage out of the box solutions for logins, account creation, registrations, and password creation. This also meant we did not have to re-engineer a security feature, which kept development costs lower. For enterprise reporting and integration, Effectual chose Microsoft SQL Server and IIS.
By consolidating expenses, utilizing spot instances, and integrating AWS services with existing Microsoft technologies, we deployed custom applications that have significantly reduced PPP’s costs and improved their overall user experience year after year.
Several years ago, Effectual worked with BBQ manufacturer MAK Grills on a product ideation project for a new web app giving owners remote operational control of their grills. Prior to our engagement, the company’s outsourced development process had stalled and they needed help salvaging the project. After reviewing their existing code, we were able to address their wishlist and launch the app on an aggressive timeline.
While the app ran successfully for the first few years, the company began to experience performance issues as its customer base expanded. MAK Grills President Bob Tucker re-engaged Effectual for a Well-Architected Framework Review to evaluate their solution, which was crashing daily and shutting down all of their grills.
For the review, Tucker had the following objectives:
Stabilize the production environment
Build an affordable solution using their existing Microsoft technology stack (.NET, SQL Server, IIS)
In our Well-Architected Review, we discovered that the company had recently hired someone to rewrite their firmware. During the rollout, the firmware had 10,000 BBQs sending messages to the MAK Grills website every 5 seconds. This increase in traffic was causing their server to crash at least once a day. They had tried to fix the issue but it was still unresolved. With owners who expected their mobile service to be available 24/7, it was clear the company had a serious customer service problem on their hands.
Actions & Recommendations from the Well-Architected Review
Analyzed the MAK Grills Microsoft server (.NET technology stack with a SQL Server on the backend) to identify what was crashing.
Consulted with their firmware developer to provide guidance on IoT best practices.
Coordinated all of the company’s outsourced engineering teams to ensure they are on the same page in terms of cost objectives and best practices for scalability and reliability.
Our team also recommended that MAK Grills capture their market metrics to understand the business value of their offering. We installed Google Analytics to evaluate customer behavior and created a company dashboard for greater visibility into user data. In addition, we suggested they evaluate switching their business model to monthly subscription pricing (versus charging a $300 upfront cost for the app at time of purchase).
Based on user feedback and customer data, the MAK Grills sales team is now testing a monthly subscription pricing model with new customers. Effectual’s remediation has stabilized a production environment that can scale automatically, and the company can focus on new product innovation to keep its customers engaged and happy.
Economic Development of Central Oregon (EDCO): Digital solutions that adapt and evolve with a growing organization
EDCO is a regional non-profit that helps companies move, start and grow in Central Oregon.
EDCO had just stepped into a rebranding process to capture its leadership role as an information and networking hub for the region’s business community. At the top of the list: redesigning and overhauling their website. Built years before on a closed, proprietary platform, their non-responsive site didn’t reflect their own progress and was time-consuming for staff to update and keep current.
“The Effectual team has the talent and creativity to respond to whatever you can dream up – so think big.”
– Brian Vierra / Venture Catalyst, EDCO-Bend Venture Conference
Content strategy informs brand messaging
Working closely with tech companies and entrepreneurs, EDCO wanted a digital presence as dynamic as its clients. The group’s new branding aligned with its long-term strategic plan but hadn’t been translated into an effective content strategy that engaged visitors. During an extensive collaboration, we developed the voice and lexicon, user personas, and calls to action that ultimately shaped brand messaging throughout the site as well as in other marketing initiatives.
78% increase – average time on site
30% increase – number of pages viewed
Content strategy process refined overall brand messaging
Discover Your Forest + US Forest Service: Strategic consulting uncovers new opportunities to engage visitors
Discover Your Forest promotes the discovery of Deschutes and Ochoco National Forests and Crooked River National Grassland by enriching the experience of visitors, building community support and creating the next generation of environmental stewards.
DYF’s new leadership team was ready to explore using digital technology to connect visitors and volunteers to its services and expand its donor base. Our discovery process uncovered strategic opportunities for integrating digital solutions that DYF hadn’t considered possible or affordable – launching them into a new phase of innovation and expansion.
Empowering visitors with easy access to information
DYF’s wanted a digital kiosk at the new Cascade Lakes Welcome Center that gave visitors simple access to trail and permitting info. Our team built a custom web app leveraging their existing US Forest Service databases, making trail and use information user-friendly and instantly available to visitors. Directly after launch, the Forest Service began evaluating the web app for regional offices in the Northwest and beyond.
Easy, 24-7 access to visitor information
Increased permitting revenue
New digital solution for Forest Service visitor services
“Partnership with Effectual helped us engage with a wider audience than we’d ever imagined. Their strategic guidance was invaluable and it shows in our final product.”
– Rika Nelson / Executive Director, Discover Your Forest
Mobile app transforms visitor engagement
As conversations evolved, Effectual encouraged DYF to look beyond the web app to a mobile solution that could engage visitors anywhere. The Forest Service had shelved the idea of a mobile app in the past due to cost and technical issues. New research and some collective problem-solving revealed that going mobile was within reach and within budget.
Simple UX makes trail and permit info easy to discover
Expanded engagement and access beyond bricks and mortar experience
Leveraged existing technology platforms with little added cost
Created a standardized, clean set of data deployable across other Forest Service locations
Deep dive business strategy delivers outstanding online experience
Last, the DYF static website needed a complete redesign to boost engagement and connect visitors and volunteers to the group’s mission. Effectual guided their team through an in-depth planning process to identify key personas and calls to action that would drive design and user experience and deliver desired outcomes.
Finely tuned UX development and design aligned with business goals
Supporting the Delivery of Early Warning Signs for Earthquakes
Effectual delivered a mission-critical solution to a Federal Government Client that ensured the delivery of early warning alert notifications for earthquakes and other natural disasters over multiple geographical locations to save lives.
This could not have been done without a Cloud-based solution to ensure a resilient system.
This Federal Government customer required a move from its on-premises infrastructure to a centralized Cloud environment. The client looked to our team to handle high availability architecture and fault tolerance to meet workloads over multiple geographical locations quickly after a natural disaster. The solution required improved resilience and redundancy capabilities, application performance, and control monitoring.
Our team built out a highly available and scalable infrastructure to meet demand in the wake of a disaster. We utilized the customer’s containerized solution and created a pipeline leveraging a GitLab Runner in Amazon Web Services (AWS) to manipulate and manage the AWS Elastic Kubernetes Service (EKS) deployments. This ensured the client’s ability to deliver early warnings for natural disasters through their application.
Our team configured Amazon CloudWatch metrics to identify a surge in traffic in the event of a disaster. This fully integrated AWS service is built with more resilience. Kubernetes was implemented to provide automated container orchestration and higher availability to reach across multiple regions.
We created a proprietary AWS-hosted Git solution to do all the linking, testing, and delivery to code. Our solution increased the rate at which the client released updates to the solution by 90%.
We deployed a GitLab Runner in conjunction with GitLab Continuous Integration to ensure all applications were provisioned through a pipeline. These necessary changes led to extreme version control and expediting developer updates.
Bird Conservation Science Enabled by Automated Monitoring and Analysis of Migration Patterns
Effectual led a Federal Government client in need of automation, reliability, and efficiency for their bird identification website.
The customer supports the collection, archiving, management and dissemination of information from banded and marked birds in North America. This information is used to monitor the status and trends of resident and migratory bird populations.
This Federal Government customer required a move from its on-premises infrastructure to a centralized cloud environment. The client looked to our team to redesign their website, creating a system that would produce automated checks to save time and manual effort when registering banded and checked birds into the database.
Our team assisted the customer in creating a system that would require minimal effort to keep up and running for years. This system saved time and manual effort through the implementation of Amazon Elastic Compute Cloud (Amazon EC2) to automate cron jobs for repetitive tasks to push all submitted web surveys from bird hunters and enthusiasts to the on-premises database. When banded birds were checked in, the system would be able to ensure the identification was correct, eliminating the need to manually check that information.
We utilized Amazon EC2 to automate database syncing. This allowed the bird banding lab to be more efficient when a bird was reported on their website. The client no longer needed to manually log and input the bird species. AWS CloudFormation was implemented to reduce manual work while developing an environment, ensuring productivity when debugging issues.
We used GitLab Continuous Integration in conjunction with GitLab Continuous Deployment to check code for errors, expediting developer changes.
Our team implemented Amazon CloudWatch Events for serverless workflow to trigger Lambda functions. Without having to provision or manage, the client was able to keep the same server running by keeping it warm with a CloudWatch Event. This reduced response times from 3 seconds to a couple hundred milliseconds.
Ensuring Least Privilege Access: Implementing an Active Directory Federation Service
Effectual led the implementation of an enterprise grade Active Directory Federation Service (ADFS) for a large Federal Government client.
Effectual enabled reliable and secure cyberspace capability by providing a highly innovative network architecture, engineering, integration, and simulation services with unrivaled expertise and commitment.
The client looked to our team to move its highly disparate environment into a highly collaborative one. By implementing Federated Access to the Amazon Web Services environment, this ensured least privilege access to client users.
We worked with the client to setup an AWS Identity and Access Management (IAM), federated sign-in through Active Directory (AD), and Active Directory Federation Services (ADFS). This ensured least privilege access to client users.
Our team enabled reliable collaborative connectivity to a cadre of remote workers that needed access to the system while utilizing the ADFS PIV card solution.
We were able to meet all security requirements by using a federated solution, allowing the client to set permissions and access levels across different systems. The Federated solution also improved auditing management of credentials.
We implemented AWS CloudFormation to create a template to use when multiple accounts register in the system. This led to an increase in efficiency and ensures consistent configurations overtime.
Serverless Infrastructure Enables Data Access Related to Environmental Issues
Effectual worked with a Federal Government client to provide a mission critical solution that scientists could apply to classification tasks on large data sets related to earthquakes, volcanic activity, waterline erosion, and water quality.
This Federal Government customer looked to our team to migrate its on-premises infrastructure to a serverless infrastructure on AWS. The client was in need of a centralized data catalog, management solution for users, and data access for environmental issues.
We supported the client with a serverless solution that consisted of Amazon API Gateway, Amazon Cognito User Pools, AWS Lambda, and AWS Step Functions. This ensured the customer’s ability to make high-volume, complex data accessible to stakeholders, policymakers, and managers to facilitate data-driven conversations about environmental issues in a secure setting.
Our team implemented API Gateway to handle the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls to process any surge of traffic on its website.
Wey implemented AWS Cognito User Pools for control over user authentication and user access for the website. This allowed for secure token handling and management of authenticated users from all identity providers.
We implemented Lambda functions to run code in a serverless environment and process its large data sets related to environmental issues. The client was able to reduce cost by 80%.
RFD & Associates, Inc., is an IT Technical Services Company with over 30 years of experience delivering IT solutions to public and private sector clients.
RFD delivers solutions from Mainframe to Mobile and everything inbetween. They have helped hundreds of organizations design, build, purchase and implement optimal technology solutions to achieve business goals. RFD needed help designing and developing a scalable, Amazon Web Services (AWS) cloud hosted, multi-tenant web and mobile friendly application. The proposed solution had a requirement to integrate with external APIs to ensure flexibility for future enhancements and integration with third-party tools. The application was also required to be compliant with Personally Identifiable Information (PII) and the U.S. Health Insurance Portability and Accountability Act (HIPAA) security.
Effectual Provided Guidance in the following areas
AWS design and architectural services to include making RFD’s multi-tenant hosting environment PII/HIPAA compliant
Provided AWS Training and best practices guidance on how to leverage AWS resources
Assisted in helping RFD achieve its defined goals:
Identify the challenges presented in third-party hosting of AWS.
Evaluate the use of cloud services to meet RFD business and technical requirements.
Determine portable containerization services.
Evaluate architectural decisions in AWS Commercial and GovCloud Regions.
A four-phased approach was developed to implement an AWS hosted environment for RFD:
Phase 1: Discovery, AWS Service Selection, and PII/HIPAA Security Requirements Determination.
Phase 2: AWS Foundation Build. Provisioned appropriate environments and access; established AWS accounts
Phase 3: AWS Service Build. Provisioned AWS services to include: EC2, Route53, S3, WAF, etc.
Phase 4: Process Documentation and Environment Review. Created AWS documentation of resources and provided reports on overall solution, security and cost.
We configured EC2 instances that are PII/HIPAA compliant ensuring adequate capacity to meet traffic demands and compute capacity. In addition, we implemented automated launch configurations to allow RFD to quickly launch and/or scale application severs in target environments in the future.
Security & Compliance
The implementation of AWS Compute, Storage, and PII and HIPAA compliant Database services to ensure the security of sensitive data used in the environment.
To maximize the functionality of many services, AWS CloudWatch was configured to help RFD set thresholds/alarms to monitor custom metrics for auto-scaling needs.
Digital Transformation Journey of a Global Restaurant Chain
A year long cloud transformation from a global network of physical data centers used to run company systems and applications.
One of the world’s largest restaurant chains offers technology services to its stores and franchisees that perform functions as diverse as POS systems, store management systems, data analysis and predictive analytics, digital advertisement and e-commerce and customer engagement platforms.
Their IT organization uses a small core staff with limited resources supported by a network of service providers and partners to fulfill key IT roles. Traditionally, the company used a global network of physical data centers to run their applications.
The company engaged with members of our team on key transformation initiatives
Need to build environments faster and with more flexibility than what traditional infrastructure in a data center could provide
Need to decouple scale up / out decisions from development cycles to speed the development process (get new apps and update apps faster)
Address local needs while maintaining global standards
Reduce CapEx to increase financial flexibility and agility
Lack of internal expertise in implementing and managing a global, enterprise public Cloud environment
Need to tie costs to specific internal projects and teams while benefiting from economies of scale
Following the completion of this engagement a prominent financial analyst commented: “The company is establishing a first-mover advantage with digital that can drive sustainable share gains in late 2017 and beyond.”
Reduced time to deploy infrastructure from 60- 120 days to minutes
Implemented Cloud automation for some workloads to enable same day deployments
Enabled global deployment of resources in AWS regions close to end customers / users to increase performance and decrease latency
20-50% cost reduction on next generation deployments for customer engagement and data warehouse projects vs. traditional models
Enabled internal bill back of resources in AWS to specific projects and teams
Following the completion of this engagement a prominent financial analyst commented: “company is establishing a first-mover advantage with digital that can drive sustainable share gains in late 2017 and beyond.”
Transforming an existing legacy environment and building it natively on AWS.
A leading social media company’s apps explore the humanness of people instead of simply quantifying how good they are, or judging how professional their work may be. Users can upload their work directly to their platform, using it to edit photos, with enhancements – including white-balancing, filters and journaling – readily available. One user documented his mother’s fight with cancer, displaying the emotions he and his family felt during a very trying time.
In 2015, the company acquired a platform that creates tangible photo books, prints and gifts for digital photos. The platform was using a non-native environment and didn’t believe it was the right strategic technology in the long term. The combined companies have more than 30 million monthly active users across its platform consuming 5 billion images. Reliable infrastructure users can depend on is essential. The company wanted to work with experts who to help them transform their existing legacy environment and build it cloud natively on AWS.
30 million monthly active users consuming 5 billion images
Reliable infrastructure users can depend on is essential. The company wanted to work with experts who to help them transform their existing non-native environment and build it cloud natively on AWS.
The companies DevOps team is very capable – they had significant experience with Chef, using it to configure their servers and developer machines – but the transformation to AWS required resources the team just didn’t have.
The transformation to AWS required resources the company’s team just didn’t have.
30 million monthly active users consuming 5 billion images.
Reliable infrastructure users can depend on is essential. The company wanted to work with experts who to help them transform their existing non-native environment and build it cloud natively on AWS.
Since migrating to AWS, the company has seen great benefits: “Operationally, AWS is faster and more configurable than what was being used,” said the VP of Engineering at the company. “The new environment is faster, more reliable, and cost efficient. All three are pretty important things!”
Most importantly, this solution had a big impact on the stability of the company’s platform and by extension, it’s brand. Previously, the platform had experienced random reboots on several occasions. The successful transition to AWS allowed them to have confidence going into the holiday season, their most important quarter of the year. Confidence in the reliability and performance of their platform allowed the company to focus on maintaining a great experience for their user community.
“The experts had both the attitude and the aptitude, and I feel as comfortable as I possibly could in having AWS as the long-term foundation for our infrastructure.”
Bring in the Experts
With the support of our experts the company was able to seamlessly migrate from its existing environment to AWS. From the start, the company was impressed with our experts knowledge and expertise. “They inspired a lot of confidence, and their team clearly had the technological expertise,” said the VP of Engineering. “We knew they could get the job done.”
Our experts leveraged their extensive experience with Vagrant and Packer to build a strong foundation for the company’s cloud native environment. This enabled the company to emulate the user experience of their members. The insights they gained helped them improve their product, streamline their operations and deliver an optimized user experience.
“Technology expertise is the number one thing we look for when hiring an outside firm,” said the VP of Engineering. “The experts had both the attitude and the aptitude, and I feel as comfortable as I possibly could in having AWS as the long-term foundation for our infrastructure.”