Create robots.txt in .Net Core application

 You can use middleware for creating robots.txt in .net core, 

Here is my code for robots.txt

Code:-

Installed Nuget package

PM> Install-Package RobotsTxtCore

Open -> startup.cs

  public void ConfigureServices(IServiceCollection services)

        {

            services.AddStaticRobotsTxt(builder =>

            builder

                .AddSection(section =>

                    section

                        .AddUserAgent("*")

                        .Allow("/")

                    )

                .AddSitemap(SiteContext.CurrentSite.SitePresentationURL + "googlesitemap.xml")

                );

}

 public void Configure(IApplicationBuilder app)

   {        app.UseRobotsTxt();  }