Madeyoum Ltd. is looking to change the way people and businesses generate video content and marketing materials.

Madeyoum required a cloud based architecture and implementation for their innovative, patent-pending video generation technology, in the quest to create an online video generation platform.

Such an architecture, needs to support robust and scalable web-services, manage complex video processing flows, stream video out of a CDN and handle a mixed solution of on-cloud machines and dedicated servers.

Madeyoum’s online technology is a complex architecture of web-servers, video processing and conversion machines, video generation algorithms and video streaming capabilities.
Its unique technology combination requires mixed usage of Linux and Windows processing machines and support for both on-cloud machines and dedicated servers.

The main challenge faced by the company was to create a scalable and robust implementation for its video processing flow technology, capable of handling variable loads.
The solution needed to allow Madeyoum to generate different kinds of videos and serve users out of different web domains while utilizing the same service infrastructure.

Madeyoum LTD partnered with Wiseman Software to lead the architecture, design and implementation of its cloud infrastructure.

Following a detailed analysis of Madeyoum’s infrastructure requirements and research of several cloud solutions, Wiseman Software recommended using AWS (Amazon Web Services) cloud infrastructure for the implementation of Madeyoum’s web-service and video processing system.

AWS were chosen because of their cloud infrastructure maturity; scaling capabilities and the ability to implement a mixed cloud and dedicated servers system and the extensive support for video hosting and streaming.

The technology chosen for the infrastructure was Java™ based Spring Framework 3.0 and Hibernate® over Tomcat deployment.

The complete implementation solution was crafted based upon Wiseman Software’s Cloud-First development methodology.

Based on the key elements of the methodology, AWS cloud infrastructure services were chosen wherever appropriate and fault tolerance plan was designed.

Click to Enlarge
Figure 1 – Schematic architecture and design of Madeyoum’s Video Generation web-service over AWS cloud services infrastructure

The main concern regarding the web front-end servers and database was to provide a scalable and robust solution, capable of serving hundreds to thousands of concurrent video creation activities.
In order to utilize computational resources to the maximum, a single video processing flow system was implemented, capable of generating different types of videos.

AWS Elastic Beanstalk Tomcat deployment platform was chosen as the service front-end for its ease of operation and built-in scalability, monitoring, notifications and health check capabilities.
Each of the web domains owned by Madeyoum, such as EstateVideoOnline.com targeting the real-estate virtual tours market, was deployed on a seperate beanstalk application stack.
The web application was built using Java™ Spring Framework 3.0 MVC platform and Hibernate 3.0 ORM infrastructure, harnessing the power of AWS ElastiCache memcached based solution, as its session caching system and Hibernate second level cache.

The main system database resides on AWS RDS MySQL deployment.
The database is configured for robustness and fault tolerance, as a multi-zone deployment, and designed for performance boosting using Read Replica servers.

Madeyoum’s video generation algorithm flow is composed of several processing steps.
The different steps include raw video and image conversion processing, video generation processing and output video conversion processing.

One of the challenges in this project was to develop the processing flow infrastructure.
The first version of the infrastructure was implemented based on AWS SQS queueing system and AWS SimpleDB.
Multiple processing machines, cloud based and dedicated, were listening on the queue, waiting for new jobs to arrive and updating SimpleDB with the processing jobs status.
Once a processor finished processing a job out of the queue, it updated the job state in the SimpleDB repository and sent AWS SNS notification for further job handling.

In order to prevent excessive polling actions over the SQS jobs queues, a solution combining SQS messages and SNS notifications was implemented.

Click to Enlarge
Figure 2 – Coordinating the processing activities by combining SQS and SNS. Processing information is stored in SimpleDB tables

The second version of the video processing flow system was implemented using Amazon Simple Workflow Service (SWF) for greater simplicity.
The new infrastructure includes EC2 cloud machines and dedicated servers deployed with Java™ Flow framework activities and workflow implementations.
Additional monitoring and maintenance jobs and workflow implementations were added using the SWF infrastructure, allowing a more robust and fault tolerant service.

Click to Enlarge
Figure 3 – Coordinating processing activities and system maintenance using AWS SWF

Madeyoum offers users the option to play and stream their generated videos to a vast selection of devices and available media players.
This capability is feasible due to the combined usage of AWS S3 for video storage with AWS CloudFront for progressive download and RTMP streaming.

Click to Enlarge
Figure 4 – Using the full strength of CloudFront CDN for RTMP streaming to Flash players & Progressive Download

Distributed system logging is hard to maintain and manage. Collecting logs from different machines and displaying a unified and complete log picture is hard and tedious.

The system combines AWS SimpleDB as its logging container and log4j Java™ library as its logging facility, using a customized log4j appender.

Madeyoum’s Admin system offers visual display of current processing flow system’s status and allows deep and extensive log querying.
These vital and important admin and monitoring features were easily implemented using AWS SDK and AWS SimpleDB query language, based on the logs being continually collected on SimpleDB repositories.

Madeyoum launched its first release for real-estate virtual tours video creation successfully and on time.
As evidence to its sophisticated and smart cloud implementation,
Madeyoum’s innovative platform and unique AWS cloud infrastructure was nominated among the leading 100 startups of the 2011 AWS Challenge.

Wiseman Software is a boutique software solution provider, specializing in software architecture, complex web application development and cloud computing implementations.
Wiseman Software provides cloud development and consulting services to software companies looking to implement their technology over the cloud.
Whether it is a complex, high availability cloud infrastructure solution or a smaller cloud migration or development project,
Wiseman Software delivers software projects on time, supporting clients in their time-to-market considerations.

Madeyoum Ltd. revolutionizes the way people and businesses create video content, unleashing the power of video commercial creation to everyone.
Madeyoum’s proprietary patent pending technology, Dynamic Storyboard, offers users the ability to self-create professional video content with a unique story and message,
using images and videos taken from any available device (such as digital cameras, camcorders, mobile phones cameras, etc.) within minutes,
Madeyoum operates a chain of market oriented websites over AWS infrastructure – such as EstateVideoOnline.com for real-estate video virtual tours creation.