Integrate Azure Pipelines With LambdaTest


Azure Pipelines is a service presented by Azure DevOps to help projects ship faster with the help of a robust pipeline. With Azure Pipelines you can easily generate a build, test it before deploying the changes live in any language you are comfortable with i.e. Node.js, Python, Java, PHP, Ruby, C/C++, .NET etc. The platform offers unlimited CI/CD minutes for open-source projects, along with 10 free parallel jobs. Azure pipelines offers great cross-platform support using which you can run jobs in parallel across Windows, macOS, Linux.

LambdaTest now integrates with Azure to boost your go-to-market delivery. Perform automated cross browser testing with LambdaTest to ensure your development code renders seamlessly through an online Selenium grid providing 2000+ real browsers running through machines, on the cloud. Perform automation testing in parallel with LambdaTest’s Selenium grid to drastically trim down your test cycles.

Pre-requisite


  1. An Azure CI account.
  2. A Git or GitHub repository.
  3. LambdaTest Authentication Credentials
  4. Be aware of your LambdaTest authentication credentials i.e. your LambdaTest username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your LambdaTest automation dashboard by clicking on the key icon near the help button.

    • For Linux/Mac:
    • For Windows:

Integrating LambdaTest with Azure Pipelines


First and foremost, you need to clone our GitHub repository for downloading the relevant code needed to integrate Azure Pipelines with LambdaTest.

Step 1: First and foremost, we need to create a project. Populate the relevant fields as you wish and hit the create button.

Create a project

Step 2: Now, Azure Pipelines will ask you about the location of your code. If you have your code repository in GitHub then select the option for GitHub from the values.

Step 3: After you choose an option for your code repository, you will be asked to login. Post login, you need to choose the particular repository. The repository will pick the azure-pipelines.yml file.

Step 4: Executing the above file will run the script in LambdaTest Grid successfully. See the below image of the output.

Parallel Testing


For running the test in parallel, replace protractor single.conf.js with protractor parallel.conf.js in same YML file.

Monitor and analyze your test result on the LambdaTest Automation Dashboard.

Deploy your code in a reliable manner at scale using Azure integration with LambdaTest, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing!

Lambda Tunnel Modifiers


This document will help you understand all the modifiers/arguments used with Lambda Tunnel to help you test your locally hosted web pages.

Command
Example
Description
-key
abcd1234efgh
Access key of the user.
-port
8080
(optional) Local port to connect tunnel.
-proxy-host
WonderProxy
Full hostname for the proxy you’d like to use.
-proxy-pass
WonderProxy_Pass
WonderProxy password of the user.
-proxy-port
22
The port declared by you for fetching the proxy.
-proxy-user
WonderProxy_Username
WonderProxy username of the user.
-tunnelName
admin12
(optional) Name of the tunnel.
-user
abc@lambdatest.com
Registered email ID of the user.

TeamCity Integration With LambdaTest


Developed by JetBrains, TeamCity is Java-based build management to help you with continuous integration and continuous deployment. With TeamCity, you can deploy your automated pipeline quicker, pacing your go-to-market launch.

LambdaTest now integrates with TeamCity to boost your go-to-market delivery. Perform automated cross browser testing with LambdaTest to ensure your development code renders seamlessly through an online Selenium grid providing 2000+ real browsers running through machines, on the cloud. Perform automation testing in parallel with LambdaTest’s Selenium grid to drastically trim down your test cycles.

Prerequisites


  • TeamCity server along with one TeamCity Agent installed.
  • Make sure team city Agent has Node and Protractor installed
  • A GIT repository.
  • LambdaTest Authentication Credentials
  • Be aware of your LambdaTest authentication credentials i.e. your LambdaTest username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your LambdaTest automation dashboard by clicking on the key icon near the help button.

    • For Linux/Mac:
      $ export LT_USERNAME= {your lambdatest username}
      $ export LT_ACCESS_KEY= {your lambdatest access_key}
    • For Windows:
      $ set LT_USERNAME= {your lambdatest username}
      $ set LT_ACCESS_KEY= {your lambdatest access_key}

Integrating LambdaTest with TeamCity


Step 1: Go to your TeamCity server URL.

Step 2: Go to Administration and create a new project manually.

create a new project manually

Step 3: After filing details and click on the Create button.

Step 4: Create a new VCS root using the below form. Required inputs are: Git Repo URL into Fetch URL and Authentication method is password where you need to pass your git username/ password.

Create a new VCS root

Step 5: Fill details, test connection and click on the Create button.

Step 6: Go to the General Settings screen for the added project. Now, create a build configuration.

build configuration

Step 7: Go to Build Step page and add a build step as below.
add a build step

Step 8: Save build step and go to Parameters and create two environment variables as below where LT_USERNAME and LT_ACCESS_KEY will be your authentication credentials for LambdaTest.

Environment Variables

Step 9: Go to Project Dashboard and Run your project. Now, notice the status of test running successfully on LambdaTest Grid with the below log.

LambdaTest Log

Running Test in Parallel


Parallel Testing is one of the most demanding features of LambdaTest Selenium Grid. By parallel testing, you can run more than one test case, simultaneously. This means that Parallel testing would allow you to execute numerous automation test cases altogether. So you execute a single test scenario across different browsers or could run different test scenarios across the same browser but with different browser versions.

Clone our GitHub repository for TeamCity to execute your first parallel test of TeamCity + LambdaTest Integration.

After you clone our GitHub repository, execute the below command in cmd/terminal.

The above command will trigger the execution of below code for performing parallel testing at LambdaTest Selenium Grid.

Kudos! Now you can Deploy your code in a reliable manner at scale using TeamCity integration with LambdaTest, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing! 🙂

White-listing Proxy With LambdaTest


If you are accessing the internet through a firewall-protected network then you may face an issue in connecting with LambdaTest. You may encounter the below error messages:

In such scenarios, you may need to whitelist LambdaTest domain. Whitelisting is a process which conveys your firewall-protected network to allow access for a set of IP addresses, IP range, or an entire domain trusted by you. Whitelisting is a security procedure to help users access a website or web-app which is considered to be non-malicious by their network security team.

You can relay the below domain to your network security team, asking them to whitelist it so you could connect with LambdaTest from your firewall protected network.

Protocol

Ports

Source

Destination

TCP

80, 443, 22

All LambdaTest Clients

*.lambdatest.com

In case of any queries or special request related to whitelisting, just give us a shout.

Changing Individual Test Details


There may be times when you wish to manipulate individual test details from your automation test build while performing automation testing with Selenium Grid on LambdaTest. This document will help you perform basic test operations for a particular test you wish to update.

Marking Test Status As Pass Or Fail


While performing automation testing with Selenium on LambdaTest Selenium Grid, you may face a scenario where a test that you declared as fail in your local instance may turn out to be completed successfully at LambdaTest. Don’t worry though! We understand how imperative it is to flag an automation test as either “pass” or “fail” depending upon your testing requirement with respect to the validation of expected behaviour.

If you wish to mark a test status as fail/pass at LambdaTest then you can do so by executing the below code:

  • To mark test status to fail.
  • To mark test status to pass.

Renaming Your Test


Sometimes, you may wish to rename your test, after placing it under execution. You can do that with the help of below code:

Semaphore CI Integration With LambdaTest


Semaphore is a hosted continuous integration and deployment service used for testing and deploying software projects hosted on GitHub and BitBucket. While open source projects can use Semaphore for free in its full capacity, free use for private projects is limited to 100 builds per month.

LambdaTest now integrates with Semaphore CI to boost your go-to-market delivery. Perform automated cross browser testing, hand in hand with your CI/CD pipeline and ensure your web-app renders seamlessly on more than 2000 real browsers and browser versions. Using LambdaTest Selenium Grid, you can fully automate your continuous testing process and trim down your test cycles, significantly.

Prerequisites For Integrating Semaphore CI With LambdaTest


We will be performing demonstration by using the Protractor framework

  1. Global Dependencies
    • A Git or GitHub repository
    • Download and install node.js and node package manager or npm.
      To install node.js with homebrew use the below command.
    • If you have npm already installed, you may want to upgrade it to latest version. Here the code you can run in your terminal to upgrade npm.
  2. LambdaTest Authentication Credentials
    • Be aware of your LambdaTest authentication credentials i.e. your LambdaTest username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your LambdaTest automation dashboard by clicking on the key icon near the help button.

Integrating LambdaTest With Semaphore CI


A few changes in the .semaphore/semaphore.yml is all you need for integrating LambdaTest With Semaphore CI. The These changes would include replacing the environment variables such as access key, username, grid config and so on, according to your test configurations.

Here is a sample config file for integrating your LambdaTest account with your Semaphore CI instacnce. You can also find the below code at our GitHub repository of Semaphore.

LambdaTest offers a Selenium Desired Capabilities Generator to fetch coded values for your desired test configurations.
For Instance, if you select the below configurations:

Fields
Selected Values
Operating System
Windows 8.1
Browser
Mozilla Firefox
Browser Version
62.0
Resolution
1280×1024
Selenium Version
3.11.0

Then Capabilities Generator at LambdaTest will provide you with the below program:

For more information, refer to our documentation on Selenium Automation Capabilities.

Testing Locally Hosted or Privately Hosted Projects


To help you perform cross browser testing of your locally stored web pages, LambdaTest provides an SSH(Secure Shell) tunnel connection with the name Lambda Tunnel. With Lambda Tunnel, you can execute a test server inside your Semaphore CI build container to perform automated cross browser testing on browsers offered by Selenium grid on LambdaTest. So you make sure how well your changes look, even before your customers. Curious to know more about Lambda Tunnel?

Follow our documentation on Lambda Tunnel to know it all.

Now, let us take an example of Semaphore.yml for Lambda Tunnel.

Once, the tunnel is successfully set up. You can add the below code to your capabilities for testing internal servers on your network.

Parallel Testing


Parallel Testing is one of the most demanding features of LambdaTest Selenium Grid. By parallel testing, you can run more than one test case, simultaneously. This means that Parallel testing would allow you to execute numerous automation test cases altogether. So you execute a single test scenario across different browsers or could run different test scenarios across the same browser but with different browser versions.

To perform automation testing in parallel on Windows use the below command.

Monitor and analyze your test result on the LambdaTest Automation Dashboard. Deploy your code in a reliable manner at scale using Semaphore CI integration with LambdaTest, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing!

Codeship Integration With LambdaTest


Codeship is a scalable, cloud-based platform for continuous integration that offers support to GitHub, Bitbucket, and GitLab projects. There are 2 versions of Codeship, Codeship Basic and Codeship Pro. Codeship Basic instantly provides robust VMs along with technology stacks being preinstalled to help run your CI/CD pipeline at a faster pace. Keep in mind though, the main purpose of Codeship Basic is to develop applications with general workflows by UI, it doesn’t offer support for dockers.

LambdaTest now integrates with Codeship to boost your go-to-market delivery. Perform automated cross browser testing with LambdaTest to ensure your development code renders seamlessly through an online Selenium grid providing 2000+ real browsers running through machines, on the cloud. Perform automation testing in parallel with LambdaTest’s Selenium grid to drastically trim down your test cycles.

Prerequisites


  1. Signing details of Codeship
  2. A GitHub repository. Here is our sample GitHub repository for Codeship Basic.
  3. LambdaTest Authentication Credentials
  4. Be aware of your LambdaTest authentication credentials i.e. your LambdaTest username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your LambdaTest automation dashboard by clicking on the key icon near the help button.

    • For Linux/Mac:
      $ export LT_USERNAME= {your lambdatest username}
      $ export LT_ACCESS_KEY= {your lambdatest access_key}
    • For Windows:
      $ set LT_USERNAME= {your lambdatest username}
      $ set LT_ACCESS_KEY= {your lambdatest access_key}

Integrating Codeship With LambdaTest


Step 1: Sign up on Codeship with your GitHub account, and create a “New Project”.

Step 2: Select GitHub as your SCM.

Step 3: Provide your GitHub organization and your GitHub repository, then hit Connect.

Step 4: Now, you need to opt for Codeship Basic as your project type by clicking on “Select Basic Project”.

Step 5: You will be redirected to a screen of Project Settings. Select the technology to prepopulate basic commands with respect to your tests.
For below demonstration, we will be using Protractor so you can go ahead and select Node.js. Now, enter the below commands in setup.

Step 6: Create a new pipeline in the section “Configure Test Pipelines”. Use the below command to connect it with your GitHub repository by replacing the GitHubOrganization & reponame with the values provided in the input by you.

Step 7: Provide your LambdaTest authentication credentials in the Environment Variables and hit “Save and go to Dashboard” button.

That is it! Now, any changes you commit in your GitHub repository will trigger the build in Codeship and run your test on LambdaTest Selenium Grid.

Testing Locally Hosted or Privately Hosted Projects


To help you perform cross browser testing of your locally stored web pages, LambdaTest provides an SSH(Secure Shell) tunnel connection with the name Lambda Tunnel. With Lambda Tunnel, you can execute a test server inside your GitLab CI build a container to perform automated cross browser testing on browsers offered by Selenium grid on LambdaTest. So you make sure how well your changes look, even before your customers.
Curious to know more about Lambda Tunnel?

Follow our documentation on Lambda Tunnel to know it all.

The below example of pipelinecommand would demonstrate you to leverage LambdaTest Selenium Grid by performing cross browser testing through the test server contained in your Codeship build.
Create two pipelines with the following commands:

Pipeline 1

Pipeline 2

Setup command will be same as mentioned earlier under the “Integration With Codeship” section. After then commit any changes into repository and get your test run using LambdaTest Tunnel.

Parallel Testing


Go to Project Settings screen. Change your pipeline command with following one & save settings.

Now, any changes commit in your repository will trigger a build and run automation tests in parallel on LambdaTest Selenium Grid.

Monitor and analyze your test result on the LambdaTest automation dashboard.

Deploy your code in a reliable manner at scale using Codeship integration with LambdaTest, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing!

Troubleshooting Lambda Tunnel


Lambda Tunnel helps in establishing an SSH connection between your local machine and our cloud servers to help you perform tests on locally hosted websites and web-apps. With Lambda Tunnel, you can ensure how robust your website rendering is across 2000+ real browsers, even before you make it live on the internet.
If due to some unfortunate reason, you are unable to perform cross browser testing using Lambda Tunnel then this document will help you troubleshoot the most common challenges.

Tunnel Not Connecting


  • Make sure you have whitelisted LambdaTest IP & port 22 isn’t blocked.
  • Verify if your network is able to connect with Lambda Tunnel by ping to the below command.
  • To debug the issue, you need to execute the below command.

Localhost Not Reachable


This issue is much prominent with Internet explorer, Edge and Safari browsers.


  • If you want to access the localhost web app through Lambda Tunnel then make sure to use “http://localhost.lambdatest.com” or enter the IP address of the local system in the URL field for avoiding any issue.

  • If you want to access the web app running on the public or private server then you need to enter the IP address of the public or private server where your web app is running.
  • If the IP address does not work then you may need to do some entries into the etc/host file as follows.
                 

    For Mac & Linux


    Step 1:Run the below command in the terminal.

    Step 2:then need to add.

    For Windows User


    You need to do same changes in 

  • If Tunnel is not working with localhost for Angular apps you need to append this command ng serve --host 0.0.0.0 --disable-host-check 
  • You can also check out this link for better understanding.

Connection Refused


If you are facing the same error as displayed in the above image(Dail: tcp Connection refused) then It implies that the traffic is reachable to Lambda tunnel, however, the URL entered is not able to reach out to localhost through the SSH tunnel.

In case if you are facing any other issue, just give us a shout and we will be happy to help. 🙂

Integrating Your Jenkins Pipeline With LambdaTest


Jenkins Pipeline is also referred to as “Pipeline” offers a suite of plugins to help integrate your continuous delivery pipeline into Jenkins. Jenkins Pipeline does so with the help of Pipeline DSL(Domain Specific Language) syntax that facilitates easy modelling of even the most complex delivery pipeline.

Prerequisites For Configuring Jenkins Pipeline With LambdaTest


  1. Jenkins 2.X or greater version.
  2. Ensure you have the Pipeline plugin, although, it is displayed under the “suggested plugins” during the post-installation setup of Jenkins.
  3. LambdaTest Authentication Credentials
  4. Be aware of your LambdaTest authentication credentials i.e. your LambdaTest username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your LambdaTest automation dashboard by clicking on the key icon near the help button.

    • For Linux/Mac:
    • For Windows:

Setting Up Jenkins Pipeline


You can find the Jenkins file for the Pipeline here or you can find the code for the Pipeline below.
Below is the code for the Jenkins Pipeline.

Refer to our Github sample repository. Now, follow the below steps for the setup.

Step 1: Create a new project for Pipeline by going to Jenkins –> New Item.

Step 2: Select the Pipeline.

Step 3: Scroll down to Advanced Project Options. Paste the Code in the code pane or fetch it via SCM & hit the Save button.

Note: To run on the tunnel, Either you can use LT_TUNNEL Environment variable to set the tunnelling capability or you can pass in the code. Instructions on the tunnel are written in Repo Readme file.

Mark As Bug In Automation Testing


Now, you can avoid the hassle of capturing a screenshot, annotating the same before you finally share them with a teammate by logging onto a respective tool for creating a task. LambdaTest does it all for you.
Mark as Bug is a feature that would help you to highlight any UI observation from your test sessions on LambdaTest and share the same with your colleagues on their favourite project management/ bug tracking tools such as Jira, Trello, Hive, Paymo etc. You can choose the assignee who handles your reported task, or bug. You can even mention the description to help them understand your point better. You can do all that while testing on LambdaTest, without having to hop on to any other platform or tool.

Note: Mark as Bug feature is not restricted to any particular feature at LambdaTest. In other words, every type of testing that you perform on LambdaTest will have mark as bug embedded in it. Below are various types of testing that we offer at LambdaTest.

  • Real Time Testing: To help you test your web-app with live interaction by running it on VMs hosted by LambdaTest cloud server.
  • Screenshot Testing: Automated bulk screenshot testing on 25(browsers + OS) combination in a single test session.
  • Responsive Testing: Perform cross browser testing across a variety of mobile & desktop devices to ensure your website renders well across different view-ports. Run resposive testing on 46 devices in a single test session.
  • Automation Testing: Perform automated cross browser testing with our on-cloud Selenium Grid. You can execute parallel testing to trim down your test cycles drastically. Our Selenium Grid supports is compatible with every test automation framework that supports Selenium.
  • Smart UI Testing: Automated visual regression testing that performs a pixel-to-pixel comparison between multiple images & highlights any UI deviations such as padding, icon, text, etc with respect to the baseline images.

Did You Know?

LambdaTest offers more than 2000 real browsers and browser versions on every type of testing so you could maximize test coverage of your cross browser testing.

How Can I Mark As Bug In Automation Testing?


Step 1: Visit the Automation tab from the left navigation menu to redirect yourself on the automation console. Go to AUTOMATION LOGS.

Automation Dashboard

Step 2: Visiting the automation logs, you will be able to access various logs of your test script execution such as network logs, command logs, and more. By default, you would be routed to SUMMARY. In summary, you will find a button to Create Issue.

Automation Dashboard

As you click on the bug icon, an integration specific form would open up. Fill the fields as per your requirement. For instance, if you have installed LambdaTest + JIRA integration then you can fill the below represented fields.

  • You can select which project should the bug go under.
  • You can select the assignee for that particular task.
  • You can select the issue type of the bug.
  • You can even post a summary to help relate the cause of the bug.

Automation Dashboard

All the fields you populate, will be directly reflected in your respective project based upon the integration you have configured with your LambdaTest account. In above case, the changes would be reflect on the respective JIRA instance.

Similary, you can find the mark as bug button in EXCEPTION and COMMAND logs.

  • Exception logs

    Exception Logs

  • Command Logs

    Command Logs

Got any questions?
Feel free to share them with our 24×7 chat support or you can drop a mail on support@lambdatest.com. Happy testing!