Changelog

Follow new updates and improvements to Checkly.

April 17th 2024

New

We just shipped our new Coralogix integration. With this integration you can:

  • Send all of your checks as Open Telemetry traces to your Coralogix account.

  • Automatically correlate the HTTP requests executed by your checks to your backend.

  • Get a link in all check results pointing to a trace in Coralogix ๐Ÿ‘‡ making detection and triaging times 10x shorter when a check fails.


For more details, check out the documentation or hop right in and navigate to the integrations page.

This feature is available on the Team and Enterprise plans.


If you have questions or feedback, join our Slack community and share your thoughts with Checkly users from across the globe.

Type @ to mention posts

April 10th 2024

New

We are excited to announce that Multistep checks are now in General Availability.

Increase reliability and confidence in your API user flows by chaining together multiple requests in a single check.

What is a Multistep Check?

Like Checklyโ€™s Browser checks, a Multistep check is a fully programmable Playwright test script at its core. It uses Playwright's API testing framework to mimic real user journeys and actions. The flexible nature of the check allows you to change, delete, or add new data in between requests, simulating any user action.

Unlike traditional multistep monitoring, you are not limited to a form builder; you can create a check that monitors exactly what you need.

Of course, Multistep checks support Checklyโ€™s monitoring-as-code workflow, not limiting you to working in our in-app editor. Work from your preferred IDE, and use our CLI to automatically update your monitors as part of your CI/CD process.

Creating your first Multistep check

To get started, simply log in to Checkly and click the โ€˜+โ€™ button in the left-hand sidebar. Select โ€˜Multistep checkโ€™ from the options and either choose a template that fits your use case, or start from scratch.

Multistep checks use Playwrights test.step method to build the check result tree. We recommend only using a single request per step in your test, and use descriptive labels for each step to make debugging and analyzing failed check runs easier. Hereโ€™s a short example.

import { test, expect } from '@playwright/test'

const baseUrl = 'https://api.myproduct.com/v1'

test('My test', async ({request}) => {
    await test.step('First step', async () => {
        const health = await request.get(`${baseUrl}/health`)
        expect(health).toBeOK()
    });

    await test.step('Second step', async () => {
        const response = await request.post(`${baseUrl}/health`)
        // Assertions for the second step
        ...
    })
})

You can test-run the script directly from the check editor to confirm it is working as intended. The left-hand sidebar contains the check results from your ad-hoc runs for easy debugging.

Multistep checks are supported on the Checkly runtime 2023.09 or later.

Degraded state and soft assertions

If you want to monitor your API for non-critical errors or performance degradations you can use the degraded check state. This allows you to signal that parts of a multistep check performed slower than expected, or that it triggered assertions that are of lower criticality.

To trigger a degraded state multistep checks use a helper library,ย @checkly/playwright-helpers, which is included in runtimeย 2023.09ย and later. The helper library contains two methods,ย markCheckAsDegradedย andย getAPIResponseTime.

Here is an example of how to use the degraded state in a check together with soft assertions from Playwright to signal that something is not performing as expected, without interrupting the execution of the check.

import { test, expect } from '@playwright/test'
import { getAPIResponseTime, markCheckAsDegraded } from "@checkly/playwright-helpers"

const baseUrl = 'https://api.myproduct.com/v1'

test('My test', async ({request}) => {
    await test.step('First step', async () => {
        const health = await request.get(`${baseUrl}/health`)
        expect(health).toBeOK()
        
        //Check if the request status is degraded
        expect.soft(getAPIResponseTime(health), 'GET /health is too  slow').toBeLessThanOrEqual(200)
    });

  // Trigger degraded state if check failed due to soft assertion
  if (test.info().errors.length) {
    markCheckAsDegraded('Check degraded due to soft assertion failure.')
  }
})

@checkly/playwright-helpers is also available for use in browser checks.

Multistep checks and monitoring as code

Multistep checks are fully supported in the Checkly monitoring as code workflow. You can use either the Checkly CLI or our Terraform provider. Here is an example of how to use the MultistepCheck construct.

import { MultiStepCheck } from 'checkly/constructs'

new MultiStepCheck('multistep-check-1', {
  name: 'Multistep Check #1',
  runtimeId: '2024.02',
  frequency: Frequency.EVERY_10M,
  locations: ['us-east-1', 'eu-west-1'],
  code: {
    entrypoint: path.join(__dirname, 'home.spec.ts')
  },
})

Availability and price

Multistep checks are available on all current Checkly price plans. Each request in a multistep check counts as a single API check run for billing purposes.

Additional resources

If you have questions or feedback, join our Slack community and share your thoughts with Checkly users from across the globe.

Happy monitoring!

Type @ to mention posts

April 3rd 2024

Improved

We are happy to announce that we have released a major update to our check overview page. It is now easier than ever to:

  • Understand the historical health of your check at a glance.

  • Find specific check results.

The new check overview chart comes with a side panel where all check results from the selected time period are visible. This allows for fast deep dives into check reports to understand and resolve incidents.

Below is an example of how navigating the old overview page could be like. Finding a result that happened in the past could be challenging, so making this a smooth experience was important to us.

With the new chart, finding a check result is simple. Each bar on the chart represents a time range and shows the aggregate of all check runs that happened in that time, with color coding to allow for fast identification of failing runs. You can use the filters on the top of the overview page to select the overall time range, and filter on locations.

Selecting a bar in the new chart further filters out the results in the sidebar, allowing you to quickly navigate to any given check result. You can also filter on the status in the chart legend to exclude a status from the results.

If you have questions or feedback, join our Slack community and share your thoughts with Checkly users from across the globe.

Happy monitoring!

Type @ to mention posts

March 26th 2024

New

Our latest runtime, 2024.02, is now GA.

The 2024.02 runtime includes several new libraries, plus updates to Playwright and other libraries. We also removed support for Mocha.

New libraries:

  • @axe-core/playwright

  • @opentelemetry/api

  • @opentelemetry/sdk-metrics

  • @opentelemetry/exporter-metrics-otlp-grpc

  • chai-json-schema

  • nice-grpc

  • pdftojson

We have also enabled the node datagram and performance modules.

With the addition of @axe-core/playwright, accessibility monitoring is now possible. Here is an introduction video on how to work with Axe and Playwright ๐Ÿ‘‡

2024.02 uses Playwright 1.42.1. For a list of all library versions, see our runtime specs documentation.

The private location agent supporting 2024.02 is v3.3.0. Pull the latest version from https://hub.docker.com/r/checkly/agent

If you have questions or feedback, join our Slack community and let us know.

Type @ to mention posts

February 14th 2024

Fixed

Improved

This patch release of the Checkly CLI fixes some small bugs and adds

Alerting additions

You can now set the alerting threshold for failing locations when using parallel scheduling. in the Web UI this configured like below ๐Ÿ‘‡

For the CLI, you add the parallelRunFailureTreshold object to your alertEscalationPolicy:

alertEscalationPolicy: AlertEscalationBuilder
    .runBasedEscalation(2,
      { amount: 2, interval: 5 }, // sets reminders
      { enabled: true, percentage: 50 } // sets the parallelRunFailureThreshold
),

Global configuration fix

We missed exporting the userAgent as an option for global configuration. This option is added now. This means two things:

1. You can set a user agent in your global config, e.g.

use: {
  userAgent: 'my funky user'
}


2. Using the devices macro now sets the correct user agent header. That field was ignored earlier, e.g.

import { devices } from '@playwright/test'

use: {
   ...devices['iPhone 12 Pro Max landscape']
}


the above code will set the user agent header to:

"Mozilla/5.0 (iPhone; CPU iPhone OS 14_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Mobile/15E148 Safari/604.1"

Learn more about the Playwright Global Configuration Beta.

To get started with the Checkly CLI just run

npm create checkly 

Or upgrade your existing installation with

npm install checkly@latest


Check the full release note on GitHub here

Type @ to mention posts

January 31st 2024

New

Parallel scheduling is now GA.

Reduce mean time to detection, get better insights when addressing outages, and get improved data granularity on all performance trends with parallel scheduling.

A check using parallel scheduling will run from each configured location each time it is scheduled, ensuring that any local outages are detected instantly. Alerts are sent as soon as a location is registered as failing or degrading, minimizing the time from failure to your team being notified.

Changing your scheduling method

With GA, parallel scheduling is now the default for all new checks and groups. To change the scheduling method, go to "Scheduling & Locations" when editing your check.

Parallel scheduling is fully supported in our monitoring as code workflow. With our CLI, just add the runParallel property to your construct:

import { ApiCheck, AssertionBuilder } from 'checkly/constructs'

new ApiCheck('hello-api-1', {
  name: 'Hello API',
  runParallel: true,
  locations: ['eu-central-1', 'us-east-1'],
  request: {
    method: 'GET',
    url: 'https://mac-demo-repo.vercel.app/api/hello',
    assertions: [
      AssertionBuilder.statusCode().equals(200)
    ],
  }
})


Similar, with Terraform just add run_parallel

resource "checkly_check" "list-all-checks" {
  name                      = "List all checks"
  type                      = "API"
  frequency                 = 0
  frequency_offset          = 10
  activated                 = false
  muted                     = false
  run_parallel              = true
  locations                 = ["eu-north-1", "eu-central-1", "us-west-1", "ap-northeast-1"]
  degraded_response_time    = 5000
  max_response_time         = 20000
  request {
    method                    = "GET"
    url                       = "https://developers.checklyhq.com/reference/getv1checks"
  }
}

Viewing check results

The check overview page will now show the progress of all locations as a check is running, giving you instant feedback if a certain location is taking longer than usual.

The check result page has been updated to support results from multiple locations. You can switch between locations in the sidebar when viewing the check result page.

Quickly compare the results from multiple locations, reviewing failures, Playwright traces or screenshots to understand how your service behaves globally.

Pricing and availability

  • Parallel check scheduling is available on all current plans: Hobby, Team, and Enterprise. Deprecated plan types are not supported.

  • Each location selected for a check running in parallel does a full check run per execution. An API check that runs in five locations will be billed for five check runs per execution. For more information on pricing, see our documentation: Check pricing.

Additional resources

Type @ to mention posts

January 26th 2024

New

We now support a subset of global configuration options for our Playwright-powered Browser and Multistep checks via our CLI.

To get started, upgrade your CLI to version 4.6.0 and add any supported options to the new playwrightConfig section in your checkly.config.ts file. To quickly copy the supported settings over to your checkly.config.ts file, you can run...

npx checkly sync-playwright


You should now have a checkly.config.ts file similar to the example below ๐Ÿ‘‡

import { defineConfig } from 'checkly'
import { devices } from '@playwright/test'

const config = defineConfig({
  projectName: 'Checkly website',
  logicalId: 'checkly-website-project',
  checks: {
    locations: ['us-east-1', 'eu-west-1'],
    playwrightConfig: {
      timeout: 30000,
      use: {
        baseURL: 'https://www.checklyhq.com',      
        ...devices['iPhone 12 Pro Max landscape'],
        extraHTTPHeaders: {
          'x-my-custom-header': 'value',        
        }
      }
    }, 
    checkMatch: ['**/__checks__/**/*.check.ts'],
    browserChecks: {
      testMatch: ['**/__checks__/**/*.spec.ts'],
    },   
  },
})

export default config

Note that we now set the timeout, baseUrl, device properties and extraHTTPHeaders globally for all Browser and Multistep checks using the canonical Playwright configuration options. You can now write a Browser check like...

import { test, expect } from '@playwright/test'

test('Checkly pricing page', async ({ page, baseURL }) => {
  const response = await page.goto(baseURL! + '/pricing')
  expect(response?.status()).toBeLessThan(400)
})


After deploying your checks with npx checkly deploy, you can access the fully rendered configuration in the new "playwright test config"-section in the web UI editor.

This is our first step to fully support all global configuration options. We still have some dragons to slay to support globalSetup, globalTeardown, storageState and dependencies but we are working it!

Check our docs for more information.

Type @ to mention posts

January 26th 2024

New

The latest release of the Checkly CLI brings the following new capabilities and improvements.

Global playwright.config.ts support in Beta

We now support a subset of global configuration options for our Playwright-powered browser and multistep checks. We also have a dedicated changelog entry with all the details.

To get started, just add any supported options to the new playwrightConfig section in your checkly.config.ts file. To quickly copy the supported settings over to your checkly.config.ts file, you can run:

npx checkly sync-playwright


You should then have a config file similar to the example below ๐Ÿ‘‡

import { defineConfig } from 'checkly'
import { devices } from '@playwright/test'

const config = defineConfig({
  projectName: 'Checkly website',
  logicalId: 'checkly-website-project',
  checks: {
    locations: ['us-east-1', 'eu-west-1'],
    playwrightConfig: {
      timeout: 30000,
      use: {
        baseURL: 'https://www.checklyhq.com',      
        ...devices['iPhone 12 Pro Max landscape'],
        extraHTTPHeaders: {
          'x-my-custom-header': 'value',        
        }
      }
    }, 
    checkMatch: ['**/__checks__/**/*.check.ts'],
    browserChecks: {
      testMatch: ['**/__checks__/**/*.spec.ts'],
    },   
  },
})

export default config

This is our first step to fully support all global configuration options. We still have some dragons to slay to support globalSetup, globalTeardown, storageState and dependencies but we are working it!

Check our docs for more information ๐Ÿ‘‡

Alert Escalation Policies

You can now manage your threshold based alerting with the new alertEscalationPolicy and it's friend the AlertEscalationBuilder.

import { MultiStepCheck, AlertEscalationBuilder } from "checkly/constructs"

const multi = new MultiStepCheck('multi-step-check-1', {
    name: 'Multi step check 1',
    locations: ['eu-west-1'],
    frequency: 10,
    alertEscalationPolicy: AlertEscalationBuilder
        .timeBasedEscalation(5, { amount: 2, interval: 5 }),
    code: {
        entrypoint: 'multi.spec.ts',
    }
})

The new alertEscalationPolicy property is available at the global project level, the CheckGroup and individual Check level. Learn more in the docs.

Other improvements

  • The checkMatch and testMatch properties now accept an array of paths so you can more easily organise your files in your repo, i.e.

    checkMatch: ['**/checks/**/*.check.ts', '**/more-checks/**/*.check.ts']
  • We now also print the test session URL to your terminal when using the dot and ci reporters.

See the all PRs merged on GitHub

To get started with the Checkly CLI just run

npm create checkly 

Or upgrade your existing installation with

npm install checkly@latest


Check the full release note on GitHub here

Type @ to mention posts

January 17th 2024

New

Visual regression & snapshot testing using Playwright is now GA.

With one line of code, you can now check visual regressions for your sites, apps and component libraries in production and get alerted when things break.

No extra tokens, wrappers or confusing config needed.

import { test, expect } from '@playwright/test';

test('Google search homepage', async ({ page }) => {
   await page.goto('https://google.com')
   await expect(page).toHaveScreenshot()
})

It's super simple:

  1. Add an expect().toHaveScreenshot() or expect().toMatchSnapshot() line to any browser check.

  2. Create a golden image with one click.

  3. Run your check around the clock. Checkly takes care of storing your images in our cloud.

Using our CLI, you can manage this all from your code base. Just add the --update-snapshots flag to your test command.

npx checkly test --update-snapshots

When your snapshot comparison fails, you can inspect it in the handy diff viewer.

To get going with snapshot testing using our CLI and / or the Checkly agent on your private location, make sure to upgrade to the latest version.

Next steps


This feature is available Team and Enterprise plans.

Questions or feedback? Join our Slack community.

Type @ to mention posts

January 12th 2024

New

All traffic from Checkly's synthetic checks (Browser, API and Multistep) is now coming from a set of static IPs.

This makes it possible to allowlist Checkly traffic in your firewalls and other network appliances and/or tools. You can fetch the range of IPs from a set of handy API endpoints to help with automating this process.

For more info on how to set this up, check out the links below:

This feature is now available across the full Checkly network architecture and is available to all users on all plans.

Questions or feedback? Join our Slack community.

Type @ to mention posts