Simeon uses Azure Pipelines to run Syncs. By default, Azure Pipelines run on Microsoft cloud-hosted agents. These agents are supplied by Microsoft, run on Windows, and are not customizable. Microsoft agents are limited to one parallel job and 1800 free minutes per month, or you can pay for unlimited minutes and an additional parallel job for $40/month.
Microsoft provides the ability to use self-hosted agents. These agents are not hosted by Microsoft but instead need to be hosted by the client using a physical server or a VM. Self-hosted agents afford more flexibility at a lower cost.
- Self-hosted agents are provided with unlimited minutes per month.
- Visual Studio Enterprise subscribers receive one free parallel job per subscriber.
- Additional parallel jobs require $15/mo billing per job.
- Self-hosted agents can be used with a static IP address and fixed location.
- Self-hosted agents can be customized with hardware and software configurations to meet your needs.
Prerequisites to use self-hosted agents with Simeon
Simeon supports Windows and Linux agents. Windows agents need to run on Windows 10 or higher. For Linux, Ubuntu is recommended.
- Debian-based distributions other than Ubuntu may work but have not been tested. Certain Linux agents require Ubuntu, such as using scale set agents.
- Other distributions based on RHEL such as Fedora, CentOS, SUSE, etc. have not been tested and may require additional configuration on the user's part and may not be supported by Simeon.
To get started, users will need the following:
- To review and understand the Windows prerequisites or Linux prerequisites.
- A VM or server running Windows or Linux with at least 2 CPU cores and 8 GB of RAM. More CPU cores and RAM allow the agent to perform more actions faster, while too few resources can crash the agent and result in failed Syncs. The number of parallel jobs, configurations being Synced, etc. will determine the hardware requirements.
- Microsoft-hosted agents run with 2 CPU cores and 7 GB RAM, for reference.
Network Requirements
Self-hosted agents must be configured to access certain resources required by Simeon and Microsoft. This includes connections to Azure DevOps, M365, and Simeon endpoints. If you are configuring a firewall on your agent or your agent's network, the following steps may be necessary.
Azure DevOps:
Azure DevOps agents must be configured to allow outbound connections to specific Microsoft IP addresses and URLs. For a list of Azure DevOps IPs and other network requirements for self-hosted agents, please review the networking requirements outlined in this guide.
M365:
Access to M365 relies on several different groups of URLs and IP addresses. For information on accessing M365, please review this guide. In addition, some providers may have specific API endpoints that must be accessible by the agent if you wish to manage those configurations. Please ensure your agent can reach the following endpoints:
- Microsoft Defender:
https://wdatpprd-us3.securitycenter.windows.com/api/
Simeon Cloud:
In addition, you will need to ensure your agent can connect to the following Simeon Cloud API management IP address:
20.112.227.84
and the Simeon Cloud GitHub package feed:
https://nuget.pkg.github.com/simeoncloud
Install Simeon Dependencies
To run Simeon, users will need to install the following software onto the agent:
- git
- Git long path support should be enabled by running the following script on the agent:
# Set git config to allow long paths
Write-Host "Setting git long paths."
git config --system core.longpaths true
- PowerShell Core
- .NET 7 SDK
- .NET 8 SDK
- NuGet
Configuring the agent
Once the agent machine has been configured, users will need to provision a user account with permissions for the agent. Directions can be found here for Windows and here for Linux.
- Create an agent pool. Navigate to DevOps Org settings Agent pools Add pool in the top right.
- Ensure the appropriate permissions have been configured for Simeon. Navigate to DevOps Org settings Agent pools the self-hosted agent security. The DevOps owner/admin as well as the [Tenants]/Project Valid Users group should be configured like this:
- Set up authentication access. There are several options outlined in this guide. Personal Access Tokens can be configured using this guide.
- Once you have a user set up with an authentication method, create the agent. Navigate to DevOps Org settings Agent pools the self-hosted agent pool created earlier agents New agent
- The process requires you to download the agent configuration script and run it on the self-hosted machine
- Select the OS and system architecture that matches your agent and download the file or copy the URL to download directly onto the machine.
Running the agent
Once the agent has been configured, it is time to run the agent.
- Users can start the agent by following this guide for Windows or this guide for Linux.
- After the agent is up and running, you should see the agent appear in the agent pool agent settings in DevOps.
Configure Simeon pipelines to use the agent pool
Via the Simeon application
Once the agent has been configured and is ready for use, navigate to the Simeon application > Install. Under Advanced Settings, there is an option to select the agent pool. You should see two standard options, in addition to your custom agent pool:
- Default: This option is an empty agent pool provided by Microsoft as a space to create your own self-hosted agents. By default, this pool has no agents and is not in use.
- Azure Pipelines: This option represents the default Microsoft-hosted Azure agent pool. Selecting this option will ensure the tenant Sync runs on the default Microsoft-hosted agent.
Select the pool from the list where your self-hosted agent has been configured. If creating the self-hosted agent under the Default pool, select this. Otherwise, select your custom agent pool. When ready, install or reinstall the tenant to confirm your selections. Your tenant will now Sync using the new self-hosted agent.
Manually via the Sync.yml file
Once the agent has been configured and is up and running, users can choose to manually select the agent by editing the Sync.yml file in Azure DevOps. To do this, navigate to DevOps > Tenants > Repos Sync.yml. The pool that the Sync uses is defined by the pool: key. By default, when using Microsoft hosted agents, the pool: key is set to as follows:
pool:
vmImage: windows-latest
pool:
name: self-hosted
When done, it should look like this:
Additional Considerations
Microsoft supports a variety of architectures for running self-hosted agents, including scale set agents, Docker agents, and AKS agents. These different architectures may require additional setup and configuration that is not covered in this guide and may not be supported by Simeon in all cases.
Scale set agents
Using scale set agents may require additional configuration. If you are looking to use a scale set agent, please follow the steps in this guide.
Docker agents
Docker agents are known to work with Simeon. If you are interested in configuring Docker agents, please see this guide.
Kubernetes / Azure Kubernetes Service (AKS)
Microsoft supports running Azure DevOps agents in AKS. For information regarding AKS agents, please see this guide.
Windows Registry Key
When running a Windows agent, you may experience the following error:
Failed to set registry key for long paths. If using a static sync agent please run the following command: "New-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem' -Name 'LongPathsEnabled' -Value 1 -PropertyType DWORD -Force".
In this case, you can run the following script to set this registry key:
if ($env:OS -like "Windows_NT*") {
Write-Host "Running on Windows, will try to set registry key for long paths."
New-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" -Name "LongPathsEnabled" -Value 1 -PropertyType DWORD -Force
}
else {
Write-Host "Not running on Windows."
}
Comments
0 comments
Please sign in to leave a comment.