ETL integration between ERP and web application
High-traffic LAMP-based portal in the automotive sector.
The customer reports difficulties in manually updating images, missed updates that cause display problems and slow page loading. They request automation of image synchronization with monitoring and an improvement in portal performance.
Automated synchronization
To improve synchronization, we have implemented the following flow:
-
Upload images from the management system to a data exchange area using an S3 bucket.
-
For each change made to the S3 bucket, a Python Lambda function is triggered to generate thumbnails.
-
The thumbnails generated by the Lambda function are saved to an S3 bucket.
Cloudwatch has been configured for monitoring with alerts and notifications via email/SMS in case of errors during execution.
Performance optimization
Based on our analysis, we noticed that the portal’s assets were being served without any optimization. To improve performance, in addition to generating thumbnails, a CDN in Cloudfront was configured for asset delivery. Furthermore, if the request comes from a mobile device, the images are further optimized by reducing the resolution using the Lambda@Edge service.
Technology stack
AWS
- S3 (Staging and image source)
- Cloudfront (cdn for image delivery)
- Lambda (thumbnail generation)
- Lambda@Edge (thumbnail generation for mobile devices)
- Cloudwatch (monitoraggio and alert)
Python
The new synchronization process has allowed to manage the update of about 160,000 images and the generation of the relative 480,000 thumbnails in one tenth of the time compared to the previous procedure.
On the portal, the loading time of pages and images has been reduced, improving the web vitals, and therefore also obtaining a benefit on SEO.
The cost of the AWS services used is less than $5 per month and includes:
- Storage space used on S3;
- Lambdas execution;
- Bandwidth used by Cloudfront for the delivery of the assets.
Join us to transform your digital future!