How do I move production robots.txt file into place on release build?

  • Page Owner: Not Set
  • Last Reviewed: 2020-04-02

I created a prod-release.txt file to overwrite the robots.txt that is used on QA site. I'm not exactly sure about the correct procedure to have setup replace the robots.txt with the prod-release.txt file.

  • Do I do this as a build process in Visual Studio?
  • Do I create a build step in Jenkins build for prod?
  • Do I set this up in Octopus?

Additional Posts

For an Azure DevOps deployment, I like to have all the files necessary for each environment in the project. Then I use a build step to copy over the appropriate file for the target environment.

For example, in a YAML azure pipeline, I use a Powershell step that looks something like this:

- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: |
      Copy-Item .\Project\License.qa.config .\Project\License.config -Force
      Copy-Item .\Project\robots.qa.txt .\Project\robots.txt  -Force

I also like to make the environment-specific files "dependent upon" on the root file, which would be the local developer's file. For example, in the .csproj, it would look like this:

    <Content Include="robots.qa.txt">
      <DependentUpon>robots.txt</DependentUpon>
    </Content>
    <Content Include="robots.prod.txt">
      <DependentUpon>robots.txt</DependentUpon>
    </Content>

In this example, robots.txt is for my local environment (not that a robots.txt file matters there), and robots.qa.txt is QA, robots.prod.txt is prod.