PointStar SingaporePointStar Singapore
  • Solutions
        • WORKPLACE COLLABORATION


        • Email and CollaborationIntegrated enterprise solutions for business emails, collaborations and productivity tools.
        • Meeting Room and ConferencingModern conferencing and communication solutions for future workplaces.
        • End User ComputingSecured and easy-to-manage solutions for end users to access applications and data – any device, anytime, anywhere.
        • SD-WAN, Network, Wi-FiSmart and secured cloud managed network solutions from small businesses to large enterprises.
        • CLOUD COMPUTING


        • Public And Hybrid CloudMulti-Cloud Solutions for all types of workloads securely and flexibly on pay-as-you-use model.
        • Data Cloud and AnalyticsData solutions that allows unification of data securely across organizations and platforms with intelligent insights.
        • Platform as a ServiceCloud platform solutions that provide a series of modular cloud services for application and web services.
        • API ManagementFull lifecycle API management platform as gateway for runtime management, policy governance, and usage analytics.
        • BUSINESS APPLICATIONS


        • ERP SolutionsEnterprise Resources Planning solutions to automate business process for better financial insights and internal controls.
        • CRM Solutions360 solutions to unify sales, marketing and customer service interactions for better customer engagement.
        • Digital Signature SolutionsLegally binding electronic signature solutions for documents signings and workflow approvals.
        • Document Management SystemsHardcopies digitization and documents approval solutions for enhanced business process automations.
        • EMERGING TECHNOLOGIES


        • AI ChatbotLifelike conversational AI with state-of-the-art virtual agent systems for businesses and customer service environments.
        • Machine LearningArtificial Intelligence (AI) solutions that enable data learning for predictive analysis and automated decision making.
        • IoTProtect your business from disruptions with secure and highly available cloud infrastructure.
        • >> MORE
  • Services
        • CLOUD TRANSFORMATION


        • Infrastructure ModernizationProvide flexible cloud infrastructure services from re-host to re-platform.
        • Application ModernizationProvide cloud native application development platforms and services for simplify and speedy software delivery.
        • Data AnalyticsOffer multi-cloud and cross-platform cloud analytics for data analysis and reportings.
        • Cloud MigrationMigration of workload and application from on-premises systems to cloud platform.
        • BUSINESS SERVICES


        • ConsultingImprove your operational performance and productivity, and adding value throughout the lifecycle.
        • Business Application IntegrationsRapidly builds end-to-end smart workflows to help automate digital processes.
        • Backup & Disaster RecoveryProvide centralized protection for environments and applications running on-premises and on-cloud.
        • Business Process AutomationProfessional services to assist businesses in streamlining and automating business process and re-shaping their work.
        • TECHNICAL SERVICES


        • Technical SupportCertified engineers in providing IT support services with commitment to Service Level Agreements.
        • Managed ServicesProvide expertise to manage businesses’ IT requirements for operational efficiency and faster resolutions.
        • Workplace ModernizationDesign, install and implement modern AV and communication solutions for all workplace environment types.
        • Installation ServicesProfessional services in IT installations and implementations.
        • PROFESSIONAL SERVICES


        • Training ServicesProfessional training services from our certified trainers, engineers and consultants.
        • Project ManagementProfessional services in managing processes and resources to ensure the success of a project delivery.
        • Change ManagementManage the transition in product, policy and process changes to achieve ROI in new technology adoptions.
        • >> MORE
  • Products
        • More Products >>

  • Partners
        • Partner With Us
        • More >>

  • Customers
        • Our customers
        • Customer Stories
  • Blog
  • About
        • OVERVIEW


        • About PointStarPointStar is one of the pioneers in cloud services in the region.
        • Our TeamMeet our team of cloud transformation leaders.
        • OUR PARTNERSHIP


        • Awards and AccreditationPointStar has been recognized many times for a great number of achievements.
        • JOIN POINTSTAR


        • Why Join UsWorking at our company goes far beyond just having a job.
        • CareersJob opportunities in PointStar.
        • EVENTS


        • EventsCheck out our upcoming events.
  • Contact Us
        • Contact Sales
        • Help & Support
        • Customer Service Portal
  • Shop
Search
  • Solutions
        • WORKPLACE COLLABORATION


        • Email and CollaborationIntegrated enterprise solutions for business emails, collaborations and productivity tools.
        • Meeting Room and ConferencingModern conferencing and communication solutions for future workplaces.
        • End User ComputingSecured and easy-to-manage solutions for end users to access applications and data – any device, anytime, anywhere.
        • SD-WAN, Network, Wi-FiSmart and secured cloud managed network solutions from small businesses to large enterprises.
        • CLOUD COMPUTING


        • Public And Hybrid CloudMulti-Cloud Solutions for all types of workloads securely and flexibly on pay-as-you-use model.
        • Data Cloud and AnalyticsData solutions that allows unification of data securely across organizations and platforms with intelligent insights.
        • Platform as a ServiceCloud platform solutions that provide a series of modular cloud services for application and web services.
        • API ManagementFull lifecycle API management platform as gateway for runtime management, policy governance, and usage analytics.
        • BUSINESS APPLICATIONS


        • ERP SolutionsEnterprise Resources Planning solutions to automate business process for better financial insights and internal controls.
        • CRM Solutions360 solutions to unify sales, marketing and customer service interactions for better customer engagement.
        • Digital Signature SolutionsLegally binding electronic signature solutions for documents signings and workflow approvals.
        • Document Management SystemsHardcopies digitization and documents approval solutions for enhanced business process automations.
        • EMERGING TECHNOLOGIES


        • AI ChatbotLifelike conversational AI with state-of-the-art virtual agent systems for businesses and customer service environments.
        • Machine LearningArtificial Intelligence (AI) solutions that enable data learning for predictive analysis and automated decision making.
        • IoTProtect your business from disruptions with secure and highly available cloud infrastructure.
        • >> MORE
  • Services
        • CLOUD TRANSFORMATION


        • Infrastructure ModernizationProvide flexible cloud infrastructure services from re-host to re-platform.
        • Application ModernizationProvide cloud native application development platforms and services for simplify and speedy software delivery.
        • Data AnalyticsOffer multi-cloud and cross-platform cloud analytics for data analysis and reportings.
        • Cloud MigrationMigration of workload and application from on-premises systems to cloud platform.
        • BUSINESS SERVICES


        • ConsultingImprove your operational performance and productivity, and adding value throughout the lifecycle.
        • Business Application IntegrationsRapidly builds end-to-end smart workflows to help automate digital processes.
        • Backup & Disaster RecoveryProvide centralized protection for environments and applications running on-premises and on-cloud.
        • Business Process AutomationProfessional services to assist businesses in streamlining and automating business process and re-shaping their work.
        • TECHNICAL SERVICES


        • Technical SupportCertified engineers in providing IT support services with commitment to Service Level Agreements.
        • Managed ServicesProvide expertise to manage businesses’ IT requirements for operational efficiency and faster resolutions.
        • Workplace ModernizationDesign, install and implement modern AV and communication solutions for all workplace environment types.
        • Installation ServicesProfessional services in IT installations and implementations.
        • PROFESSIONAL SERVICES


        • Training ServicesProfessional training services from our certified trainers, engineers and consultants.
        • Project ManagementProfessional services in managing processes and resources to ensure the success of a project delivery.
        • Change ManagementManage the transition in product, policy and process changes to achieve ROI in new technology adoptions.
        • >> MORE
  • Products
        • More Products >>

  • Partners
        • Partner With Us
        • More >>

  • Customers
        • Our customers
        • Customer Stories
  • Blog
  • About
        • OVERVIEW


        • About PointStarPointStar is one of the pioneers in cloud services in the region.
        • Our TeamMeet our team of cloud transformation leaders.
        • OUR PARTNERSHIP


        • Awards and AccreditationPointStar has been recognized many times for a great number of achievements.
        • JOIN POINTSTAR


        • Why Join UsWorking at our company goes far beyond just having a job.
        • CareersJob opportunities in PointStar.
        • EVENTS


        • EventsCheck out our upcoming events.
  • Contact Us
        • Contact Sales
        • Help & Support
        • Customer Service Portal
  • Shop
Home » Digital Transformation Rockstar Blog » Automate complex tasks with Google Workflows

Automate complex tasks with Google Workflows

Automate complex tasks with Google Workflows

Syah Ismail2021-08-01T23:22:03+08:00
Syah Ismail Big Data, Blog 0 Comments

BLOG

Automate complex tasks with Google Workflows

  • January 28, 2021
  • 9:23 am

Nowadays, it takes a lot of work to run a company. Whether your company is processing e-commerce transactions, producing goods or delivering IT services, you need to manage the flow of work across a variety of systems. While it’s possible to manage those workflows manually or with general-purpose tools, doing so is much easier with a purpose-built product. 

Google Cloud has two workflow tools in its portfolio: Cloud Composer and the new Workflows. Workflows is a fully managed workflow orchestration product running as part of Google Cloud. It’s fully serverless and requires no infrastructure management.

A sample workflow

First, consider the following workflow for generating an invoice:

A common way to orchestrate these steps is to call API services based on Cloud Functions, Cloud Run or a public SaaS API like SendGrid which sends an e-mail with a PDF attachment. However, real-life scenarios are typically much more complex than the example above and require continuous tracking of all workflow executions, error handling, decision points and conditional jumps, iterating arrays of entries, data conversions and many other advanced features. 

Which is to say, while technically you can use general-purpose tools to manage this process, it’s not ideal. For example, let’s consider some of the challenges you’d face processing this flow with an event-based compute platform like Cloud Functions. First, the max duration of a Cloud Function run is nine minutes but workflows especially those involving human interactions can run for days. Your workflow may need more time to complete or you may need to pause in between steps when polling for a response status. Attempting to chain multiple Cloud Functions together with, for instance, Pub/Sub also works but there’s no simple way to develop or operate such a workflow. First, in this model it’s very hard to associate step failures with workflow executions, making troubleshooting very difficult. Also, understanding the state of all workflow executions requires a custom-built tracking model, further increasing the complexity of this architecture. 

In contrast, workflow products provide support for exception handling and give visibility on executions and the state of individual steps including successes and failures. Because the state of each step is individually managed, the workflow engine can seamlessly recover from errors, significantly improving the reliability of the applications that use the workflows. Lastly, workflow products often come with built-in connectors to popular APIs and cloud products, saving time and letting you plug into existing API interfaces. 

Cloud Composer vs Workflows

Google Cloud’s first general-purpose workflow orchestration tool was Cloud Composer. Based on Apache Airflow, Cloud Composer is great for data engineering pipelines like ETL orchestration, big data processing or machine learning workflows and integrates well with data products like BigQuery or Dataflow . For example, Cloud Composer is a natural choice if your workflow needs to run a series of jobs in a data warehouse or big data cluster and save results to a storage bucket.

However, if you want to process events or chain APIs in a serverless way or have workloads that are bursty or latency-sensitive, it is recommended to use Workflows. 

Workflows scales to zero when you’re not using it, incurring no costs when it’s idle. Pricing is based on the number of steps in the workflow, so you only pay if your workflow runs. Since Workflows doesn’t charge based on execution time, if a workflow pauses for a few hours in between tasks, you don’t pay for this either. 

Workflows scale up automatically with very low startup time and no “cold start” effect. Also, it transitions quickly between steps, supporting latency-sensitive applications. 

Workflows use case

When it comes to the number of processes and flows that Workflows can orchestrate, the sky’s the limit. 

Processing customer transactions

Imagine you need to process customer orders and in the case that an item is out of stock, trigger an inventory refill from an external supplier. During order processing, you also want to notify your sales reps about large customer orders. Sales reps are more likely to react quickly if they get such notifications using Slack. 

Here is an example workflow diagram.

The workflow above orchestrates calls to Google Cloud’s Firestore as well as external APIs including Slack, SendGrid or the inventory supplier’s custom API. It passes the data between the steps and implements decision points that execute steps conditionally, depending on other APIs’ outputs. 

Each workflow execution is logged so you can trace it back or troubleshoot it if needed. The workflow handles necessary retries or exceptions thrown by APIs, thus improving the reliability of the entire application. 

Workflows under the hood

Let’s take a deeper look at some key features you’ll find in Workflows. 

Steps

Workflows handles sequencing of activities delivered as ‘steps’. If needed, a workflow can also be configured to pause between steps without generating time-related charges.

In particular, you can orchestrate practically any API that is network-reachable and follows HTTP as a workflow step. You can make a call to any internet-based API including SaaS APIs or your private endpoints without having to wrap such calls in Cloud Functions or Cloud Run.

Authentication

When making calls to Google Cloud APIs, e.g., to invoke a Cloud function or read data from Firestore, Workflows uses built-in IAM authentication. As long as your workflow has been granted IAM permission to use a particular Google Cloud API, you don’t need to worry about authentication protocols.

Communication between workflow steps

Most real-life workflows require that steps communicate with one another. Workflows supports built-in variables that steps can use to pass the result of their work to a subsequent step. 

Automatic JSON conversion

As JSON is very common in API integrations, Workflows automatically converts API JSON responses to dictionaries, making it easy for the following steps to access this information. 

Rich expression language

Workflows also comes with a rich expression language supporting arithmetic and logical operators, arrays, dictionaries and many other features. The ability to perform basic data manipulations directly in the workflow further simplifies API integrations. Since Workflows accepts runtime arguments, you can use a single workflow to react to different events or input data.

Decision points

With variables and expressions, we can implement another critical component of most workflows: decision points. Workflows can use custom expressions to decide whether to jump to another part of the workflow or conditionally execute a step. 

Conditional step execution

Frequently used parts of the logic can be coded as a sub-workflow and then called as a regular step, working similarly to routines in many programming languages.

Sometimes, a step in a workflow fails, maybe due to a network issue or because a particular API is down. This, however, shouldn’t immediately make the entire workflow execution fail. 

Workflows avoids that problem with a combination of configurable retries and exception handling that together allow a workflow to react appropriately to an error returned by the API call.

All features above are configurable as part of the Workflows source code.

Workflows is a powerful new addition to Google Cloud’s application development and management toolset, and you can try it out immediately on all your projects. 
Have a look at the Workflows site or go right ahead to the Cloud Console to build your first workflow. Workflows comes with a free tier so you can give it a try at no cost.

Professional Services for Google Cloud Platform

Choose from consulting to turn-key IT managed services for Google Cloud Platform.
Learn More

Share this post

Facebook
Twitter
LinkedIn
Email

Share this post

Facebook Twitter LinkedIn Google + Email

Author

Syah Ismail

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

POINTSTAR SINGAPORE | A CLOUD TRANSFORMATION COMPANY
PointStar Singapore is a leading cloud transformation company based in Singapore that brings businesses great solutions, with a presence across Asia including Malaysia and Indonesia. We offer cutting-edge cloud solutions like email & collaboration, video conferencing, AI chatbot, and machine learning featuring a wide range of products including Google Cloud Platform, Google Workspace, Google Workspace for Education, Google Maps Platform, Oracle NetSuite, Cisco Meraki, AppSheet, Apigee, HelloSign, and Logitech. Furthermore, we enhance these solutions with top-notch services such as infrastructure modernization, installation, change management, and technical support which means you get the best value for your investment. All because we value you as our customer. What are you waiting for? Start your transformation journey by getting a complimentary consultation from us.

Solutions

  • Email And Collaboration
  • Room And Conference
  • Public And Hybrid Cloud
  • API Management
  • CRM Solutions
  • Document Management Systems

Services

  • Cloud Migration
  • Data Analytics
  • Workspace Modernization
  • Managed Services
  • Training Services
  • Technical Support

Partners

  • Google Cloud
  • Oracle NetSuite
  • Logitech
  • Meraki
  • Freshworks
  • Microsoft

About Us

  • Our Team
  • Awards And Accreditation
  • Our Offices
  • Careers
  • Events
Copyright © 2009-2023 PointStar Pte Ltd. All Rights Reserved. Privacy Policy.
PointStar Malaysia PointStar Indonesia PointStar Consulting Alomos e-Store
Facebook Linkedin