PointStar SingaporePointStar Singapore
  • Solutions
        • WORKPLACE COLLABORATION


        • Email and CollaborationIntegrated enterprise solutions for business emails, collaborations and productivity tools.
        • Meeting Room and ConferencingModern conferencing and communication solutions for future workplaces.
        • End User ComputingSecured and easy-to-manage solutions for end users to access applications and data – any device, anytime, anywhere.
        • SD-WAN, Network, Wi-FiSmart and secured cloud managed network solutions from small businesses to large enterprises.
        • CLOUD COMPUTING


        • Public And Hybrid CloudMulti-Cloud Solutions for all types of workloads securely and flexibly on pay-as-you-use model.
        • Data Cloud and AnalyticsData solutions that allows unification of data securely across organizations and platforms with intelligent insights.
        • Platform as a ServiceCloud platform solutions that provide a series of modular cloud services for application and web services.
        • API ManagementFull lifecycle API management platform as gateway for runtime management, policy governance, and usage analytics.
        • BUSINESS APPLICATIONS


        • ERP SolutionsEnterprise Resources Planning solutions to automate business process for better financial insights and internal controls.
        • CRM Solutions360 solutions to unify sales, marketing and customer service interactions for better customer engagement.
        • Digital Signature SolutionsLegally binding electronic signature solutions for documents signings and workflow approvals.
        • Document Management SystemsHardcopies digitization and documents approval solutions for enhanced business process automations.
        • EMERGING TECHNOLOGIES


        • AI ChatbotLifelike conversational AI with state-of-the-art virtual agent systems for businesses and customer service environments.
        • Machine LearningArtificial Intelligence (AI) solutions that enable data learning for predictive analysis and automated decision making.
        • IoTProtect your business from disruptions with secure and highly available cloud infrastructure.
        • >> MORE
  • Services
        • CLOUD TRANSFORMATION


        • Infrastructure ModernizationProvide flexible cloud infrastructure services from re-host to re-platform.
        • Application ModernizationProvide cloud native application development platforms and services for simplify and speedy software delivery.
        • Data AnalyticsOffer multi-cloud and cross-platform cloud analytics for data analysis and reportings.
        • Cloud MigrationMigration of workload and application from on-premises systems to cloud platform.
        • BUSINESS SERVICES


        • ConsultingImprove your operational performance and productivity, and adding value throughout the lifecycle.
        • Business Application IntegrationsRapidly builds end-to-end smart workflows to help automate digital processes.
        • Backup & Disaster RecoveryProvide centralized protection for environments and applications running on-premises and on-cloud.
        • Business Process AutomationProfessional services to assist businesses in streamlining and automating business process and re-shaping their work.
        • TECHNICAL SERVICES


        • Technical SupportCertified engineers in providing IT support services with commitment to Service Level Agreements.
        • Managed ServicesProvide expertise to manage businesses’ IT requirements for operational efficiency and faster resolutions.
        • Workplace ModernizationDesign, install and implement modern AV and communication solutions for all workplace environment types.
        • Installation ServicesProfessional services in IT installations and implementations.
        • PROFESSIONAL SERVICES


        • Training ServicesProfessional training services from our certified trainers, engineers and consultants.
        • Project ManagementProfessional services in managing processes and resources to ensure the success of a project delivery.
        • Change ManagementManage the transition in product, policy and process changes to achieve ROI in new technology adoptions.
        • >> MORE
  • Products
        • More Products >>

  • Partners
        • Partner With Us
        • More >>

  • Customers
        • Our customers
        • Customer Stories
  • Blog
  • About
        • OVERVIEW


        • About PointStarPointStar is one of the pioneers in cloud services in the region.
        • Our TeamMeet our team of cloud transformation leaders.
        • OUR PARTNERSHIP


        • Awards and AccreditationPointStar has been recognized many times for a great number of achievements.
        • JOIN POINTSTAR


        • Why Join UsWorking at our company goes far beyond just having a job.
        • CareersJob opportunities in PointStar.
        • EVENTS


        • EventsCheck out our upcoming events.
  • Contact Us
        • Contact Sales
        • Help & Support
        • Customer Service Portal
  • Shop
Search
  • Solutions
        • WORKPLACE COLLABORATION


        • Email and CollaborationIntegrated enterprise solutions for business emails, collaborations and productivity tools.
        • Meeting Room and ConferencingModern conferencing and communication solutions for future workplaces.
        • End User ComputingSecured and easy-to-manage solutions for end users to access applications and data – any device, anytime, anywhere.
        • SD-WAN, Network, Wi-FiSmart and secured cloud managed network solutions from small businesses to large enterprises.
        • CLOUD COMPUTING


        • Public And Hybrid CloudMulti-Cloud Solutions for all types of workloads securely and flexibly on pay-as-you-use model.
        • Data Cloud and AnalyticsData solutions that allows unification of data securely across organizations and platforms with intelligent insights.
        • Platform as a ServiceCloud platform solutions that provide a series of modular cloud services for application and web services.
        • API ManagementFull lifecycle API management platform as gateway for runtime management, policy governance, and usage analytics.
        • BUSINESS APPLICATIONS


        • ERP SolutionsEnterprise Resources Planning solutions to automate business process for better financial insights and internal controls.
        • CRM Solutions360 solutions to unify sales, marketing and customer service interactions for better customer engagement.
        • Digital Signature SolutionsLegally binding electronic signature solutions for documents signings and workflow approvals.
        • Document Management SystemsHardcopies digitization and documents approval solutions for enhanced business process automations.
        • EMERGING TECHNOLOGIES


        • AI ChatbotLifelike conversational AI with state-of-the-art virtual agent systems for businesses and customer service environments.
        • Machine LearningArtificial Intelligence (AI) solutions that enable data learning for predictive analysis and automated decision making.
        • IoTProtect your business from disruptions with secure and highly available cloud infrastructure.
        • >> MORE
  • Services
        • CLOUD TRANSFORMATION


        • Infrastructure ModernizationProvide flexible cloud infrastructure services from re-host to re-platform.
        • Application ModernizationProvide cloud native application development platforms and services for simplify and speedy software delivery.
        • Data AnalyticsOffer multi-cloud and cross-platform cloud analytics for data analysis and reportings.
        • Cloud MigrationMigration of workload and application from on-premises systems to cloud platform.
        • BUSINESS SERVICES


        • ConsultingImprove your operational performance and productivity, and adding value throughout the lifecycle.
        • Business Application IntegrationsRapidly builds end-to-end smart workflows to help automate digital processes.
        • Backup & Disaster RecoveryProvide centralized protection for environments and applications running on-premises and on-cloud.
        • Business Process AutomationProfessional services to assist businesses in streamlining and automating business process and re-shaping their work.
        • TECHNICAL SERVICES


        • Technical SupportCertified engineers in providing IT support services with commitment to Service Level Agreements.
        • Managed ServicesProvide expertise to manage businesses’ IT requirements for operational efficiency and faster resolutions.
        • Workplace ModernizationDesign, install and implement modern AV and communication solutions for all workplace environment types.
        • Installation ServicesProfessional services in IT installations and implementations.
        • PROFESSIONAL SERVICES


        • Training ServicesProfessional training services from our certified trainers, engineers and consultants.
        • Project ManagementProfessional services in managing processes and resources to ensure the success of a project delivery.
        • Change ManagementManage the transition in product, policy and process changes to achieve ROI in new technology adoptions.
        • >> MORE
  • Products
        • More Products >>

  • Partners
        • Partner With Us
        • More >>

  • Customers
        • Our customers
        • Customer Stories
  • Blog
  • About
        • OVERVIEW


        • About PointStarPointStar is one of the pioneers in cloud services in the region.
        • Our TeamMeet our team of cloud transformation leaders.
        • OUR PARTNERSHIP


        • Awards and AccreditationPointStar has been recognized many times for a great number of achievements.
        • JOIN POINTSTAR


        • Why Join UsWorking at our company goes far beyond just having a job.
        • CareersJob opportunities in PointStar.
        • EVENTS


        • EventsCheck out our upcoming events.
  • Contact Us
        • Contact Sales
        • Help & Support
        • Customer Service Portal
  • Shop
Home » Digital Transformation Rockstar Blog » 6 ways to scale your serverless applications

6 ways to scale your serverless applications

6 ways to scale your serverless applications

Syah Ismail2019-12-05T14:56:57+08:00
Syah Ismail Blog, GCP 0 Comments

Imagine your service depends on APIs or databases with rate or connection limits. A spike of traffic might cause your service to scale and quickly overrun those limits. In this post, you’ll learn about features of Cloud Functions, Google Cloud’s event-driven serverless compute service and products like Cloud Tasks that can help serverless services play nice with the rest of your stack.

Serverless scaling basics

At a basic level, a function takes input and provides an output response. That function can be repeated with many inputs, providing many outputs. A serverless platform like Cloud Functions manages elastic, horizontal scaling of function instances. Because Google Cloud can provide a near-infinite scale, that can have consequences for other systems with which your serverless function interacts.

Most scale-related problems are the result of limits on infrastructure resources and time. Not all things scale the same way and not all serverless workloads have the same expected behaviours in terms of how they get work done. For example, whether or not the result of a function is returned to the caller or is only directed elsewhere, can change how you handle increasing scale in your function. Different situations may call for one or more different strategies to manage challenges scale can introduce.

Luckily, you have lots of different tools and techniques at your disposal to help ensure that your serverless applications scale effectively.

1. Use Max Instances to manage connection limits

Because serverless compute products like Cloud Functions and Cloud Run are stateless, many functions use a database like Cloud SQL for stateful data. But this database might only be able to handle 100 concurrent connections. Under modest load (fewer than 100 queries per second), this works fine. But a sudden spike can result in hundreds of concurrent connections from your functions, leading to degraded performance or outages. 

One way to mitigate this is to configure instance scaling limits on your functions. Cloud Functions offers the max instances setting. This feature limits how many concurrent instances of your function are running and attempting to establish database connections. So, if your database can only handle 100 concurrent connections, you might set max instances to a lower value, say 75. Since each instance of a function can only handle a single request at a time, this effectively means that you can only handle 75 concurrent requests at any given time.  

2. Use Cloud Tasks to limit the rate of work done

Sometimes the limit you are worried about isn’t the number of concurrent connections but the rate at which work is performed. For example, imagine you need to call an external API for which you have a limited per-minute quota. Cloud Tasks gives you options in managing the way in which work gets done. It allows you to perform the work outside of the serverless handler in one or more work queues. Cloud Tasks supports rate and concurrency limits, making sure that regardless of the rate work arrives, it is performed with rates applied. 

3. Use stateful storage to defer results from long-running operations

Sometimes you want your function to be capable of deferring the requested work until after you provide an initial response. But you still want to make the result of the work available to the caller eventually. For example, it may not make sense to try to encode a large video file inside a serverless instance. You could use Cloud Tasks if the caller of your workload only needs to know that the request was submitted. But if you want the caller to be able to retrieve some status or eventual result, you need an additional stateful system to track the job. In Google APIs this pattern is referred to as a long-running operation. There are several ways you can achieve this with serverless infrastructure on Google Cloud, such as using a combination of Cloud Functions, Cloud Pub/Sub, and Firestore.

4. Use Redis to rate limit usage

Sometimes you need to perform rate-limiting in the context of the HTTP request. This may be because you are performing per-user rate limits or need to provide a back-pressure signal to the caller of your serverless workload. Because each serverless instance is stateless and has no knowledge of how many other instances may also be serving requests, you need a high-performance shared counter mechanism. Redis is a common choice for rate-limiting implementations. Read more about rate limiting and GCP and see this tutorial for how to use serverless VPC access to reach a private Redis instance and perform rate-limiting for serverless instances.

5. Use Cloud Pub/Sub to process work in batches

When dealing with a large number of messages, you may not want to process every message individually. A common pattern is to wait until a sufficient number of messages have accumulated before handling all of them in one batch. Cloud Functions integrates seamlessly with Cloud Pub/Sub as a trigger source but serverless workloads can also use Cloud Pub/Sub as a place to accumulate batches of work as the service will store messages for up to seven days.

Then, you can use Cloud Scheduler to handle these accumulated items on a regular schedule, triggering a function that processes all the accumulated messages in one batch run. 

You can also trigger the batch process more dynamically based on the number and age of accumulated messages. Check out this tutorial which uses Cloud Pub/Sub, Stackdriver Alerting and Cloud Functions to process a batch of messages. 

6. Use Cloud Run for heavily I/O-bound work

One of the more expensive components of many infrastructure products is compute cycles. This is reflected in the pricing of many managed services which include how many time-units of CPU you use. When your serverless workload is just waiting around for a remote API call it may make to return or waiting for a file to read, these are moments where you are not using the CPU but are still “occupying it” so will be billed.  Cloud Run, which lets you run fully managed serverless containers, allows your workload to specify how many concurrent requests it can handle. This can lead to significant increases in efficiency for I/O bound workloads. 

For example, if the work being done spends most of its time waiting for replies from slow remote API calls, Cloud Run supports up to 80 requests concurrently on the same serverless instance which shares the use of the same CPU allocation. 

Just let it scale

Even if you don’t have any scaling problems with your serverless workload, you may still be uneasy, especially if this is your first time building software in a serverless environment. Rest easy, the default limits for Google Cloud serverless infrastructure are high enough to accommodate most workloads without having to do anything.

GCP Managed Services

If you do find yourself approaching those limits, we are happy to work with you to keep things running at any scale.

Learn More

Share this post

Facebook Twitter LinkedIn Google + Email

Author

Syah Ismail

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

POINTSTAR SINGAPORE | A CLOUD TRANSFORMATION COMPANY
PointStar Singapore is a leading cloud transformation company based in Singapore that brings businesses great solutions, with a presence across Asia including Malaysia and Indonesia. We offer cutting-edge cloud solutions like email & collaboration, video conferencing, AI chatbot, and machine learning featuring a wide range of products including Google Cloud Platform, Google Workspace, Google Workspace for Education, Google Maps Platform, Oracle NetSuite, Cisco Meraki, AppSheet, Apigee, HelloSign, and Logitech. Furthermore, we enhance these solutions with top-notch services such as infrastructure modernization, installation, change management, and technical support which means you get the best value for your investment. All because we value you as our customer. What are you waiting for? Start your transformation journey by getting a complimentary consultation from us.

Solutions

  • Email And Collaboration
  • Room And Conference
  • Public And Hybrid Cloud
  • API Management
  • CRM Solutions
  • Document Management Systems

Services

  • Cloud Migration
  • Data Analytics
  • Workspace Modernization
  • Managed Services
  • Training Services
  • Technical Support

Partners

  • Google Cloud
  • Oracle NetSuite
  • Logitech
  • Meraki
  • Freshworks
  • Microsoft

About Us

  • Our Team
  • Awards And Accreditation
  • Our Offices
  • Careers
  • Events
Copyright © 2009-2023 PointStar Pte Ltd. All Rights Reserved. Privacy Policy.
PointStar Malaysia PointStar Indonesia PointStar Consulting Alomos e-Store
Facebook Linkedin