Cloud Runner v0 - Reliable and trimmed down cloud runner (#353)

* Update cloud-runner-aws-pipeline.yml * Update cloud-runner-k8s-pipeline.yml * yarn build * yarn build * correct branch ref * correct branch ref passed to target repo * Create k8s-tests.yml * Delete k8s-tests.yml * correct branch ref passed to target repo * correct branch ref passed to target repo * Always describe AWS tasks for now, because unstable error handling * Remove unused tree commands * Use lfs guid sum * Simple override cache push * Simple override cache push and pull override to allow pure cloud storage driven caching * Removal of early branch (breaks lfs caching) * Remove unused tree commands * Update action.yml * Update action.yml * Support cache and input override commands as input + full support custom hooks * Increase k8s timeout * replace filename being appended for unknclear reason * cache key should not contain whitespaces * Always try and deploy rook for k8s * Apply k8s files for rook * Update action.yml * Apply k8s files for rook * Apply k8s files for rook * cache test and action description for kuber storage class * Correct test and implement dependency health check and start * GCP-secret run, cache key * lfs smudge set explicit and undo explicit * Run using external secret provider to speed up input * Update cloud-runner-aws-pipeline.yml * Add nodejs as build step dependency * Add nodejs as build step dependency * Cloud Runner Tests must be specified to capture logs from cloud runner for tests * Cloud Runner Tests must be specified to capture logs from cloud runner for tests * Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs * Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs * Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs * Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs * Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs * better defaults for new inputs * better defaults * merge latest * force build update * use npm n to update node in unity builder * use npm n to update node in unity builder * use npm n to update node in unity builder * correct new line * quiet zipping * quiet zipping * default secrets for unity username and password * default secrets for unity username and password * ls active directory before lfs install * Get cloud runner secrets from * Get cloud runner secrets from * Cleanup setup of default secrets * Various fixes * Cleanup setup of default secrets * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * AWS secrets manager support * less caching logs * default k8s storage class to pd-standard * more readable build commands * Capture aws exit code 1 reliably * Always replace /head from branch * k8s default storage class to standard-rwo * cleanup * further cleanup input * further cleanup input * further cleanup input * further cleanup input * further cleanup input * folder sizes to inspect caching * dir command for local cloud runner test * k8s wait for pending because pvc will not create earlier * prefer k8s standard storage * handle empty string as cloud runner cluster input * local-system is now used for cloud runner test implementation AND correctly unset test CLI input * local-system is now used for cloud runner test implementation AND correctly unset test CLI input * fix unterminated quote * fix unterminated quote * do not share build parameters in tests - in cloud runner this will cause conflicts with resouces of the same name * remove head and heads from branch prefix * fix reversed caching direction of cache-push * fixes * fixes * fixes * cachePull cli * fixes * fixes * fixes * fixes * fixes * order cache test to be first * order cache test to be first * fixes * populate cache key instead of using branch * cleanup cli * garbage-collect-aws cli can iterate over aws resources and cli scans all ts files * import cli methods * import cli files explicitly * import cli files explicitly * import cli files explicitly * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * log parameters in cloud runner parameter test * log parameters in cloud runner parameter test * log parameters in cloud runner parameter test * Cloud runner param test before caching because we have a fast local cache test now * Using custom build path relative to repo root rather than project root * aws-garbage-collect at end of pipeline * aws-garbage-collect do not actually delete anything for now - just list * remove some legacy du commands * Update cloud-runner-aws-pipeline.yml * log contents after cache pull and fix some scenarios with duplicate secrets * log contents after cache pull and fix some scenarios with duplicate secrets * log contents after cache pull and fix some scenarios with duplicate secrets * PR comments * Replace guid with uuid package * use fileExists lambda instead of stat to check file exists in caching * build failed results in core error message * Delete sample.txt
2022-04-11 00:00:37 +01:00
parent 2b399b2641
commit a61c02481f
65 changed files with 10262 additions and 2259 deletions
--- a/src/model/cloud-runner/providers/aws/aws-base-stack.ts
+++ b/src/model/cloud-runner/providers/aws/aws-base-stack.ts
@@ -0,0 +1,106 @@
+import CloudRunnerLogger from '../../services/cloud-runner-logger';
+import * as core from '@actions/core';
+import * as SDK from 'aws-sdk';
+import * as fs from 'fs';
+import path from 'path';
+const crypto = require('crypto');
+
+export class AWSBaseStack {
+  constructor(baseStackName: string) {
+    this.baseStackName = baseStackName;
+  }
+  private baseStackName: string;
+
+  async setupBaseStack(CF: SDK.CloudFormation) {
+    const baseStackName = this.baseStackName;
+
+    const baseStack = fs.readFileSync(path.join(__dirname, 'cloud-formations', 'base-setup.yml'), 'utf8');
+
+    // Cloud Formation Input
+    const describeStackInput: SDK.CloudFormation.DescribeStacksInput = {
+      StackName: baseStackName,
+    };
+    const parametersWithoutHash: SDK.CloudFormation.Parameter[] = [
+      { ParameterKey: 'EnvironmentName', ParameterValue: baseStackName },
+    ];
+    const parametersHash = crypto
+      .createHash('md5')
+      .update(baseStack + JSON.stringify(parametersWithoutHash))
+      .digest('hex');
+    const parameters: SDK.CloudFormation.Parameter[] = [
+      ...parametersWithoutHash,
+      ...[{ ParameterKey: 'Version', ParameterValue: parametersHash }],
+    ];
+    const updateInput: SDK.CloudFormation.UpdateStackInput = {
+      StackName: baseStackName,
+      TemplateBody: baseStack,
+      Parameters: parameters,
+      Capabilities: ['CAPABILITY_IAM'],
+    };
+    const createStackInput: SDK.CloudFormation.CreateStackInput = {
+      StackName: baseStackName,
+      TemplateBody: baseStack,
+      Parameters: parameters,
+      Capabilities: ['CAPABILITY_IAM'],
+    };
+
+    const stacks = await CF.listStacks({
+      StackStatusFilter: ['UPDATE_COMPLETE', 'CREATE_COMPLETE', 'ROLLBACK_COMPLETE'],
+    }).promise();
+    const stackNames = stacks.StackSummaries?.map((x) => x.StackName) || [];
+    const stackExists: Boolean = stackNames.includes(baseStackName) || false;
+    const describeStack = async () => {
+      return await CF.describeStacks(describeStackInput).promise();
+    };
+    try {
+      if (!stackExists) {
+        CloudRunnerLogger.log(`${baseStackName} stack does not exist (${JSON.stringify(stackNames)})`);
+        await CF.createStack(createStackInput).promise();
+        CloudRunnerLogger.log(`created stack (version: ${parametersHash})`);
+      }
+      const CFState = await describeStack();
+      let stack = CFState.Stacks?.[0];
+      if (!stack) {
+        throw new Error(`Base stack doesn't exist, even after creation, stackExists check: ${stackExists}`);
+      }
+      const stackVersion = stack.Parameters?.find((x) => x.ParameterKey === 'Version')?.ParameterValue;
+
+      if (stack.StackStatus === 'CREATE_IN_PROGRESS') {
+        await CF.waitFor('stackCreateComplete', describeStackInput).promise();
+      }
+
+      if (stackExists) {
+        CloudRunnerLogger.log(`Base stack exists (version: ${stackVersion}, local version: ${parametersHash})`);
+        if (parametersHash !== stackVersion) {
+          CloudRunnerLogger.log(`Attempting update of base stack`);
+          try {
+            await CF.updateStack(updateInput).promise();
+          } catch (error: any) {
+            if (error['message'].includes('No updates are to be performed')) {
+              CloudRunnerLogger.log(`No updates are to be performed`);
+            } else {
+              CloudRunnerLogger.log(`Update Failed (Stack name: ${baseStackName})`);
+              CloudRunnerLogger.log(error['message']);
+            }
+            CloudRunnerLogger.log(`Continuing...`);
+          }
+        } else {
+          CloudRunnerLogger.log(`No update required`);
+        }
+        stack = (await describeStack()).Stacks?.[0];
+        if (!stack) {
+          throw new Error(
+            `Base stack doesn't exist, even after updating and creation, stackExists check: ${stackExists}`,
+          );
+        }
+        if (stack.StackStatus === 'UPDATE_IN_PROGRESS') {
+          await CF.waitFor('stackUpdateComplete', describeStackInput).promise();
+        }
+      }
+      CloudRunnerLogger.log('base stack is now ready');
+    } catch (error) {
+      core.error(JSON.stringify(await describeStack(), undefined, 4));
+      throw error;
+    }
+  }
+}
--- a/src/model/cloud-runner/providers/aws/aws-cloud-formation-templates.ts
+++ b/src/model/cloud-runner/providers/aws/aws-cloud-formation-templates.ts
@@ -0,0 +1,38 @@
+import * as fs from 'fs';
+
+export class AWSCloudFormationTemplates {
+  public static getParameterTemplate(p1) {
+    return `
+  ${p1}:
+    Type: String
+    Default: ''
+`;
+  }
+
+  public static getSecretTemplate(p1) {
+    return `
+  ${p1}Secret:
+    Type: AWS::SecretsManager::Secret
+    Properties:
+      Name: '${p1}'
+      SecretString: !Ref ${p1}
+`;
+  }
+
+  public static getSecretDefinitionTemplate(p1, p2) {
+    return `
+            - Name: '${p1}'
+              ValueFrom: !Ref ${p2}Secret
+`;
+  }
+
+  public static insertAtTemplate(template, insertionKey, insertion) {
+    const index = template.search(insertionKey) + insertionKey.length + '\n'.length;
+    template = [template.slice(0, index), insertion, template.slice(index)].join('');
+    return template;
+  }
+
+  public static readTaskCloudFormationTemplate(): string {
+    return fs.readFileSync(`${__dirname}/cloud-formations/task-def-formation.yml`, 'utf8');
+  }
+}
--- a/src/model/cloud-runner/providers/aws/aws-error.ts
+++ b/src/model/cloud-runner/providers/aws/aws-error.ts
@@ -0,0 +1,16 @@
+import CloudRunnerLogger from '../../services/cloud-runner-logger';
+import * as SDK from 'aws-sdk';
+import * as core from '@actions/core';
+import CloudRunner from '../../cloud-runner';
+
+export class AWSError {
+  static async handleStackCreationFailure(error: any, CF: SDK.CloudFormation, taskDefStackName: string) {
+    CloudRunnerLogger.log('aws error: ');
+    core.error(JSON.stringify(error, undefined, 4));
+    if (CloudRunner.buildParameters.cloudRunnerIntegrationTests) {
+      CloudRunnerLogger.log('Getting events and resources for task stack');
+      const events = (await CF.describeStackEvents({ StackName: taskDefStackName }).promise()).StackEvents;
+      CloudRunnerLogger.log(JSON.stringify(events, undefined, 4));
+    }
+  }
+}
--- a/src/model/cloud-runner/providers/aws/aws-job-stack.ts
+++ b/src/model/cloud-runner/providers/aws/aws-job-stack.ts
@@ -0,0 +1,141 @@
+import * as SDK from 'aws-sdk';
+import CloudRunnerAWSTaskDef from './cloud-runner-aws-task-def';
+import CloudRunnerSecret from '../../services/cloud-runner-secret';
+import { AWSCloudFormationTemplates } from './aws-cloud-formation-templates';
+import CloudRunnerLogger from '../../services/cloud-runner-logger';
+import { AWSError } from './aws-error';
+
+export class AWSJobStack {
+  private baseStackName: string;
+  constructor(baseStackName: string) {
+    this.baseStackName = baseStackName;
+  }
+
+  public async setupCloudFormations(
+    CF: SDK.CloudFormation,
+    buildGuid: string,
+    image: string,
+    entrypoint: string[],
+    commands: string,
+    mountdir: string,
+    workingdir: string,
+    secrets: CloudRunnerSecret[],
+  ): Promise<CloudRunnerAWSTaskDef> {
+    const taskDefStackName = `${this.baseStackName}-${buildGuid}`;
+    let taskDefCloudFormation = AWSCloudFormationTemplates.readTaskCloudFormationTemplate();
+    for (const secret of secrets) {
+      secret.ParameterKey = `${buildGuid.replace(/[^\dA-Za-z]/g, '')}${secret.ParameterKey.replace(
+        /[^\dA-Za-z]/g,
+        '',
+      )}`;
+      if (typeof secret.ParameterValue == 'number') {
+        secret.ParameterValue = `${secret.ParameterValue}`;
+      }
+      if (!secret.ParameterValue || secret.ParameterValue === '') {
+        secrets = secrets.filter((x) => x !== secret);
+        continue;
+      }
+      taskDefCloudFormation = AWSCloudFormationTemplates.insertAtTemplate(
+        taskDefCloudFormation,
+        'p1 - input',
+        AWSCloudFormationTemplates.getParameterTemplate(secret.ParameterKey),
+      );
+      taskDefCloudFormation = AWSCloudFormationTemplates.insertAtTemplate(
+        taskDefCloudFormation,
+        'p2 - secret',
+        AWSCloudFormationTemplates.getSecretTemplate(`${secret.ParameterKey}`),
+      );
+      taskDefCloudFormation = AWSCloudFormationTemplates.insertAtTemplate(
+        taskDefCloudFormation,
+        'p3 - container def',
+        AWSCloudFormationTemplates.getSecretDefinitionTemplate(secret.EnvironmentVariable, secret.ParameterKey),
+      );
+    }
+    const secretsMappedToCloudFormationParameters = secrets.map((x) => {
+      return { ParameterKey: x.ParameterKey.replace(/[^\dA-Za-z]/g, ''), ParameterValue: x.ParameterValue };
+    });
+    const parameters = [
+      {
+        ParameterKey: 'EnvironmentName',
+        ParameterValue: this.baseStackName,
+      },
+      {
+        ParameterKey: 'ImageUrl',
+        ParameterValue: image,
+      },
+      {
+        ParameterKey: 'ServiceName',
+        ParameterValue: taskDefStackName,
+      },
+      {
+        ParameterKey: 'Command',
+        ParameterValue: 'echo "this template should be overwritten when running a task"',
+      },
+      {
+        ParameterKey: 'EntryPoint',
+        ParameterValue: entrypoint.join(','),
+      },
+      {
+        ParameterKey: 'WorkingDirectory',
+        ParameterValue: workingdir,
+      },
+      {
+        ParameterKey: 'EFSMountDirectory',
+        ParameterValue: mountdir,
+      },
+      ...secretsMappedToCloudFormationParameters,
+    ];
+
+    let previousStackExists = true;
+    while (previousStackExists) {
+      previousStackExists = false;
+      const stacks = await CF.listStacks().promise();
+      if (!stacks.StackSummaries) {
+        throw new Error('Faild to get stacks');
+      }
+      for (let index = 0; index < stacks.StackSummaries.length; index++) {
+        const element = stacks.StackSummaries[index];
+        if (element.StackName === taskDefStackName && element.StackStatus !== 'DELETE_COMPLETE') {
+          previousStackExists = true;
+          CloudRunnerLogger.log(`Previous stack still exists: ${JSON.stringify(element)}`);
+        }
+      }
+    }
+
+    try {
+      await CF.createStack({
+        StackName: taskDefStackName,
+        TemplateBody: taskDefCloudFormation,
+        Capabilities: ['CAPABILITY_IAM'],
+        Parameters: parameters,
+      }).promise();
+      CloudRunnerLogger.log('Creating cloud runner job');
+      await CF.waitFor('stackCreateComplete', { StackName: taskDefStackName }).promise();
+    } catch (error) {
+      await AWSError.handleStackCreationFailure(
+        error,
+        CF,
+        taskDefStackName,
+        //taskDefCloudFormation,
+        //parameters,
+        //secrets,
+      );
+      throw error;
+    }
+
+    const taskDefResources = (
+      await CF.describeStackResources({
+        StackName: taskDefStackName,
+      }).promise()
+    ).StackResources;
+
+    const baseResources = (await CF.describeStackResources({ StackName: this.baseStackName }).promise()).StackResources;
+
+    return {
+      taskDefStackName,
+      taskDefCloudFormation,
+      taskDefResources,
+      baseResources,
+    };
+  }
+}
--- a/src/model/cloud-runner/providers/aws/aws-task-runner.ts
+++ b/src/model/cloud-runner/providers/aws/aws-task-runner.ts
@@ -0,0 +1,236 @@
+import * as AWS from 'aws-sdk';
+import CloudRunnerEnvironmentVariable from '../../services/cloud-runner-environment-variable';
+import * as core from '@actions/core';
+import CloudRunnerAWSTaskDef from './cloud-runner-aws-task-def';
+import * as zlib from 'zlib';
+import CloudRunnerLogger from '../../services/cloud-runner-logger';
+import { Input } from '../../..';
+import CloudRunner from '../../cloud-runner';
+import { CloudRunnerStatics } from '../../cloud-runner-statics';
+import { CloudRunnerBuildCommandProcessor } from '../../services/cloud-runner-build-command-process';
+
+class AWSTaskRunner {
+  static async runTask(
+    taskDef: CloudRunnerAWSTaskDef,
+    ECS: AWS.ECS,
+    CF: AWS.CloudFormation,
+    environment: CloudRunnerEnvironmentVariable[],
+    buildGuid: string,
+    commands: string,
+  ) {
+    const cluster = taskDef.baseResources?.find((x) => x.LogicalResourceId === 'ECSCluster')?.PhysicalResourceId || '';
+    const taskDefinition =
+      taskDef.taskDefResources?.find((x) => x.LogicalResourceId === 'TaskDefinition')?.PhysicalResourceId || '';
+    const SubnetOne =
+      taskDef.baseResources?.find((x) => x.LogicalResourceId === 'PublicSubnetOne')?.PhysicalResourceId || '';
+    const SubnetTwo =
+      taskDef.baseResources?.find((x) => x.LogicalResourceId === 'PublicSubnetTwo')?.PhysicalResourceId || '';
+    const ContainerSecurityGroup =
+      taskDef.baseResources?.find((x) => x.LogicalResourceId === 'ContainerSecurityGroup')?.PhysicalResourceId || '';
+    const streamName =
+      taskDef.taskDefResources?.find((x) => x.LogicalResourceId === 'KinesisStream')?.PhysicalResourceId || '';
+
+    const task = await ECS.runTask({
+      cluster,
+      taskDefinition,
+      platformVersion: '1.4.0',
+      overrides: {
+        containerOverrides: [
+          {
+            name: taskDef.taskDefStackName,
+            environment,
+            command: ['-c', CloudRunnerBuildCommandProcessor.ProcessCommands(commands, CloudRunner.buildParameters)],
+          },
+        ],
+      },
+      launchType: 'FARGATE',
+      networkConfiguration: {
+        awsvpcConfiguration: {
+          subnets: [SubnetOne, SubnetTwo],
+          assignPublicIp: 'ENABLED',
+          securityGroups: [ContainerSecurityGroup],
+        },
+      },
+    }).promise();
+    const taskArn = task.tasks?.[0].taskArn || '';
+    CloudRunnerLogger.log('Cloud runner job is starting');
+    await AWSTaskRunner.waitUntilTaskRunning(ECS, taskArn, cluster);
+    CloudRunnerLogger.log(
+      `Cloud runner job status is running ${(await AWSTaskRunner.describeTasks(ECS, cluster, taskArn))?.lastStatus}`,
+    );
+    const output = await this.streamLogsUntilTaskStops(ECS, CF, taskDef, cluster, taskArn, streamName);
+    const taskData = await AWSTaskRunner.describeTasks(ECS, cluster, taskArn);
+    const exitCode = taskData.containers?.[0].exitCode;
+    const wasSuccessful = exitCode === 0 || (exitCode === undefined && taskData.lastStatus === 'RUNNING');
+    if (wasSuccessful) {
+      CloudRunnerLogger.log(`Cloud runner job has finished successfully`);
+      return output;
+    } else {
+      if (taskData.stoppedReason === 'Essential container in task exited' && exitCode === 1) {
+        throw new Error('Container exited with code 1');
+      }
+      const message = `Cloud runner job exit code ${exitCode}`;
+      taskData.overrides = undefined;
+      taskData.attachments = undefined;
+      CloudRunnerLogger.log(`${message} ${JSON.stringify(taskData, undefined, 4)}`);
+      throw new Error(message);
+    }
+  }
+
+  private static async waitUntilTaskRunning(ECS: AWS.ECS, taskArn: string, cluster: string) {
+    try {
+      await ECS.waitFor('tasksRunning', { tasks: [taskArn], cluster }).promise();
+    } catch (error_) {
+      const error = error_ as Error;
+      await new Promise((resolve) => setTimeout(resolve, 3000));
+      CloudRunnerLogger.log(
+        `Cloud runner job has ended ${
+          (await AWSTaskRunner.describeTasks(ECS, cluster, taskArn)).containers?.[0].lastStatus
+        }`,
+      );
+
+      core.setFailed(error);
+      core.error(error);
+    }
+  }
+
+  static async describeTasks(ECS: AWS.ECS, clusterName: string, taskArn: string) {
+    const tasks = await ECS.describeTasks({
+      cluster: clusterName,
+      tasks: [taskArn],
+    }).promise();
+    if (tasks.tasks?.[0]) {
+      return tasks.tasks?.[0];
+    } else {
+      throw new Error('No task found');
+    }
+  }
+
+  static async streamLogsUntilTaskStops(
+    ECS: AWS.ECS,
+    CF: AWS.CloudFormation,
+    taskDef: CloudRunnerAWSTaskDef,
+    clusterName: string,
+    taskArn: string,
+    kinesisStreamName: string,
+  ) {
+    const kinesis = new AWS.Kinesis();
+    const stream = await AWSTaskRunner.getLogStream(kinesis, kinesisStreamName);
+    let iterator = await AWSTaskRunner.getLogIterator(kinesis, stream);
+
+    const logBaseUrl = `https://${Input.region}.console.aws.amazon.com/cloudwatch/home?region=${CF.config.region}#logsV2:log-groups/log-group/${taskDef.taskDefStackName}`;
+    CloudRunnerLogger.log(`You can also see the logs at AWS Cloud Watch: ${logBaseUrl}`);
+    let shouldReadLogs = true;
+    let timestamp: number = 0;
+    let output = '';
+    while (shouldReadLogs) {
+      await new Promise((resolve) => setTimeout(resolve, 1500));
+      const taskData = await AWSTaskRunner.describeTasks(ECS, clusterName, taskArn);
+      ({ timestamp, shouldReadLogs } = AWSTaskRunner.checkStreamingShouldContinue(taskData, timestamp, shouldReadLogs));
+      ({ iterator, shouldReadLogs, output } = await AWSTaskRunner.handleLogStreamIteration(
+        kinesis,
+        iterator,
+        shouldReadLogs,
+        taskDef,
+        output,
+      ));
+    }
+    return output;
+  }
+
+  private static async handleLogStreamIteration(
+    kinesis: AWS.Kinesis,
+    iterator: string,
+    shouldReadLogs: boolean,
+    taskDef: CloudRunnerAWSTaskDef,
+    output: string,
+  ) {
+    const records = await kinesis
+      .getRecords({
+        ShardIterator: iterator,
+      })
+      .promise();
+    iterator = records.NextShardIterator || '';
+    ({ shouldReadLogs, output } = AWSTaskRunner.logRecords(records, iterator, taskDef, shouldReadLogs, output));
+    return { iterator, shouldReadLogs, output };
+  }
+
+  private static checkStreamingShouldContinue(taskData: AWS.ECS.Task, timestamp: number, shouldReadLogs: boolean) {
+    if (taskData?.lastStatus === 'UNKNOWN') {
+      CloudRunnerLogger.log('## Cloud runner job unknwon');
+    }
+    if (taskData?.lastStatus !== 'RUNNING') {
+      if (timestamp === 0) {
+        CloudRunnerLogger.log('## Cloud runner job stopped, streaming end of logs');
+        timestamp = Date.now();
+      }
+      if (timestamp !== 0 && Date.now() - timestamp > 30000) {
+        CloudRunnerLogger.log('## Cloud runner status is not RUNNING for 30 seconds, last query for logs');
+        shouldReadLogs = false;
+      }
+      CloudRunnerLogger.log(`## Status of job: ${taskData.lastStatus}`);
+    }
+    return { timestamp, shouldReadLogs };
+  }
+
+  private static logRecords(
+    records,
+    iterator: string,
+    taskDef: CloudRunnerAWSTaskDef,
+    shouldReadLogs: boolean,
+    output: string,
+  ) {
+    if (records.Records.length > 0 && iterator) {
+      for (let index = 0; index < records.Records.length; index++) {
+        const json = JSON.parse(
+          zlib.gunzipSync(Buffer.from(records.Records[index].Data as string, 'base64')).toString('utf8'),
+        );
+        if (json.messageType === 'DATA_MESSAGE') {
+          for (let logEventsIndex = 0; logEventsIndex < json.logEvents.length; logEventsIndex++) {
+            let message = json.logEvents[logEventsIndex].message;
+            if (json.logEvents[logEventsIndex].message.includes(`---${CloudRunner.buildParameters.logId}`)) {
+              CloudRunnerLogger.log('End of log transmission received');
+              shouldReadLogs = false;
+            } else if (message.includes('Rebuilding Library because the asset database could not be found!')) {
+              core.warning('LIBRARY NOT FOUND!');
+            } else if (message.includes('Build succeeded')) {
+              core.setOutput('build-result', 'success');
+            } else if (message.includes('Build fail')) {
+              core.setOutput('build-result', 'failed');
+              core.error('BUILD FAILED!');
+            }
+            message = `[${CloudRunnerStatics.logPrefix}] ${message}`;
+            if (CloudRunner.buildParameters.cloudRunnerIntegrationTests) {
+              output += message;
+            }
+            CloudRunnerLogger.log(message);
+          }
+        }
+      }
+    }
+    return { shouldReadLogs, output };
+  }
+
+  private static async getLogStream(kinesis: AWS.Kinesis, kinesisStreamName: string) {
+    return await kinesis
+      .describeStream({
+        StreamName: kinesisStreamName,
+      })
+      .promise();
+  }
+
+  private static async getLogIterator(kinesis: AWS.Kinesis, stream) {
+    return (
+      (
+        await kinesis
+          .getShardIterator({
+            ShardIteratorType: 'TRIM_HORIZON',
+            StreamName: stream.StreamDescription.StreamName,
+            ShardId: stream.StreamDescription.Shards[0].ShardId,
+          })
+          .promise()
+      ).ShardIterator || ''
+    );
+  }
+}
+export default AWSTaskRunner;
--- a/src/model/cloud-runner/providers/aws/cloud-formations/base-setup.yml
+++ b/src/model/cloud-runner/providers/aws/cloud-formations/base-setup.yml
@@ -0,0 +1,391 @@
+AWSTemplateFormatVersion: '2010-09-09'
+Description: AWS Fargate cluster that can span public and private subnets. Supports
+  public facing load balancers, private internal load balancers, and
+  both internal and external service discovery namespaces.
+Parameters:
+  EnvironmentName:
+    Type: String
+    Default: development
+    Description: 'Your deployment environment: DEV, QA , PROD'
+  Version:
+    Type: String
+    Description: 'hash of template'
+
+  # ContainerPort:
+  #   Type: Number
+  #   Default: 80
+  #   Description: What port number the application inside the docker container is binding to
+
+Mappings:
+  # Hard values for the subnet masks. These masks define
+  # the range of internal IP addresses that can be assigned.
+  # The VPC can have all IP's from 10.0.0.0 to 10.0.255.255
+  # There are four subnets which cover the ranges:
+  #
+  # 10.0.0.0 - 10.0.0.255
+  # 10.0.1.0 - 10.0.1.255
+  # 10.0.2.0 - 10.0.2.255
+  # 10.0.3.0 - 10.0.3.255
+
+  SubnetConfig:
+    VPC:
+      CIDR: '10.0.0.0/16'
+    PublicOne:
+      CIDR: '10.0.0.0/24'
+    PublicTwo:
+      CIDR: '10.0.1.0/24'
+
+Resources:
+  # VPC in which containers will be networked.
+  # It has two public subnets, and two private subnets.
+  # We distribute the subnets across the first two available subnets
+  # for the region, for high availability.
+  VPC:
+    Type: AWS::EC2::VPC
+    Properties:
+      EnableDnsSupport: true
+      EnableDnsHostnames: true
+      CidrBlock: !FindInMap ['SubnetConfig', 'VPC', 'CIDR']
+
+  EFSServerSecurityGroup:
+    Type: AWS::EC2::SecurityGroup
+    Properties:
+      GroupName: 'efs-server-endpoints'
+      GroupDescription: Which client ip addrs are allowed to access EFS server
+      VpcId: !Ref 'VPC'
+      SecurityGroupIngress:
+        - IpProtocol: tcp
+          FromPort: 2049
+          ToPort: 2049
+          SourceSecurityGroupId: !Ref ContainerSecurityGroup
+          #CidrIp: !FindInMap ['SubnetConfig', 'VPC', 'CIDR']
+  # A security group for the containers we will run in Fargate.
+  # Rules are added to this security group based on what ingress you
+  # add for the cluster.
+  ContainerSecurityGroup:
+    Type: AWS::EC2::SecurityGroup
+    Properties:
+      GroupName: 'task security group'
+      GroupDescription: Access to the Fargate containers
+      VpcId: !Ref 'VPC'
+      # SecurityGroupIngress:
+      #   - IpProtocol: tcp
+      #     FromPort: !Ref ContainerPort
+      #     ToPort: !Ref ContainerPort
+      #     CidrIp: 0.0.0.0/0
+      SecurityGroupEgress:
+        - IpProtocol: -1
+          FromPort: 2049
+          ToPort: 2049
+          CidrIp: '0.0.0.0/0'
+
+  # Two public subnets, where containers can have public IP addresses
+  PublicSubnetOne:
+    Type: AWS::EC2::Subnet
+    Properties:
+      AvailabilityZone: !Select
+        - 0
+        - Fn::GetAZs: !Ref 'AWS::Region'
+      VpcId: !Ref 'VPC'
+      CidrBlock: !FindInMap ['SubnetConfig', 'PublicOne', 'CIDR']
+      #  MapPublicIpOnLaunch: true
+
+  PublicSubnetTwo:
+    Type: AWS::EC2::Subnet
+    Properties:
+      AvailabilityZone: !Select
+        - 1
+        - Fn::GetAZs: !Ref 'AWS::Region'
+      VpcId: !Ref 'VPC'
+      CidrBlock: !FindInMap ['SubnetConfig', 'PublicTwo', 'CIDR']
+      #  MapPublicIpOnLaunch: true
+
+  # Setup networking resources for the public subnets. Containers
+  # in the public subnets have public IP addresses and the routing table
+  # sends network traffic via the internet gateway.
+  InternetGateway:
+    Type: AWS::EC2::InternetGateway
+  GatewayAttachement:
+    Type: AWS::EC2::VPCGatewayAttachment
+    Properties:
+      VpcId: !Ref 'VPC'
+      InternetGatewayId: !Ref 'InternetGateway'
+
+  # Attaching a Internet Gateway to route table makes it public.
+  PublicRouteTable:
+    Type: AWS::EC2::RouteTable
+    Properties:
+      VpcId: !Ref 'VPC'
+  PublicRoute:
+    Type: AWS::EC2::Route
+    DependsOn: GatewayAttachement
+    Properties:
+      RouteTableId: !Ref 'PublicRouteTable'
+      DestinationCidrBlock: '0.0.0.0/0'
+      GatewayId: !Ref 'InternetGateway'
+
+  # Attaching a public route table makes a subnet public.
+  PublicSubnetOneRouteTableAssociation:
+    Type: AWS::EC2::SubnetRouteTableAssociation
+    Properties:
+      SubnetId: !Ref PublicSubnetOne
+      RouteTableId: !Ref PublicRouteTable
+  PublicSubnetTwoRouteTableAssociation:
+    Type: AWS::EC2::SubnetRouteTableAssociation
+    Properties:
+      SubnetId: !Ref PublicSubnetTwo
+      RouteTableId: !Ref PublicRouteTable
+
+  # ECS Resources
+  ECSCluster:
+    Type: AWS::ECS::Cluster
+
+  # A role used to allow AWS Autoscaling to inspect stats and adjust scaleable targets
+  # on your AWS account
+  AutoscalingRole:
+    Type: AWS::IAM::Role
+    Properties:
+      AssumeRolePolicyDocument:
+        Statement:
+          - Effect: Allow
+            Principal:
+              Service: [application-autoscaling.amazonaws.com]
+            Action: ['sts:AssumeRole']
+      Path: /
+      Policies:
+        - PolicyName: service-autoscaling
+          PolicyDocument:
+            Statement:
+              - Effect: Allow
+                Action:
+                  - 'application-autoscaling:*'
+                  - 'cloudwatch:DescribeAlarms'
+                  - 'cloudwatch:PutMetricAlarm'
+                  - 'ecs:DescribeServices'
+                  - 'ecs:UpdateService'
+                Resource: '*'
+
+  # This is an IAM role which authorizes ECS to manage resources on your
+  # account on your behalf, such as updating your load balancer with the
+  # details of where your containers are, so that traffic can reach your
+  # containers.
+  ECSRole:
+    Type: AWS::IAM::Role
+    Properties:
+      AssumeRolePolicyDocument:
+        Statement:
+          - Effect: Allow
+            Principal:
+              Service: [ecs.amazonaws.com]
+            Action: ['sts:AssumeRole']
+      Path: /
+      Policies:
+        - PolicyName: ecs-service
+          PolicyDocument:
+            Statement:
+              - Effect: Allow
+                Action:
+                  # Rules which allow ECS to attach network interfaces to instances
+                  # on your behalf in order for awsvpc networking mode to work right
+                  - 'ec2:AttachNetworkInterface'
+                  - 'ec2:CreateNetworkInterface'
+                  - 'ec2:CreateNetworkInterfacePermission'
+                  - 'ec2:DeleteNetworkInterface'
+                  - 'ec2:DeleteNetworkInterfacePermission'
+                  - 'ec2:Describe*'
+                  - 'ec2:DetachNetworkInterface'
+
+                  # Rules which allow ECS to update load balancers on your behalf
+                  # with the information sabout how to send traffic to your containers
+                  - 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer'
+                  - 'elasticloadbalancing:DeregisterTargets'
+                  - 'elasticloadbalancing:Describe*'
+                  - 'elasticloadbalancing:RegisterInstancesWithLoadBalancer'
+                  - 'elasticloadbalancing:RegisterTargets'
+                Resource: '*'
+
+  # This is a role which is used by the ECS tasks themselves.
+  ECSTaskExecutionRole:
+    Type: AWS::IAM::Role
+    Properties:
+      AssumeRolePolicyDocument:
+        Statement:
+          - Effect: Allow
+            Principal:
+              Service: [ecs-tasks.amazonaws.com]
+            Action: ['sts:AssumeRole']
+      Path: /
+      Policies:
+        - PolicyName: AmazonECSTaskExecutionRolePolicy
+          PolicyDocument:
+            Statement:
+              - Effect: Allow
+                Action:
+                  # Allow the use of secret manager
+                  - 'secretsmanager:GetSecretValue'
+                  - 'kms:Decrypt'
+
+                  # Allow the ECS Tasks to download images from ECR
+                  - 'ecr:GetAuthorizationToken'
+                  - 'ecr:BatchCheckLayerAvailability'
+                  - 'ecr:GetDownloadUrlForLayer'
+                  - 'ecr:BatchGetImage'
+
+                  # Allow the ECS tasks to upload logs to CloudWatch
+                  - 'logs:CreateLogStream'
+                  - 'logs:PutLogEvents'
+                Resource: '*'
+
+  DeleteCFNLambdaExecutionRole:
+    Type: 'AWS::IAM::Role'
+    Properties:
+      AssumeRolePolicyDocument:
+        Version: '2012-10-17'
+        Statement:
+          - Effect: 'Allow'
+            Principal:
+              Service: ['lambda.amazonaws.com']
+            Action: 'sts:AssumeRole'
+      Path: '/'
+      Policies:
+        - PolicyName: DeleteCFNLambdaExecutionRole
+          PolicyDocument:
+            Version: '2012-10-17'
+            Statement:
+              - Effect: 'Allow'
+                Action:
+                  - 'logs:CreateLogGroup'
+                  - 'logs:CreateLogStream'
+                  - 'logs:PutLogEvents'
+                Resource: 'arn:aws:logs:*:*:*'
+              - Effect: 'Allow'
+                Action:
+                  - 'cloudformation:DeleteStack'
+                  - 'kinesis:DeleteStream'
+                  - 'secretsmanager:DeleteSecret'
+                  - 'kinesis:DescribeStreamSummary'
+                  - 'logs:DeleteLogGroup'
+                  - 'logs:DeleteSubscriptionFilter'
+                  - 'ecs:DeregisterTaskDefinition'
+                  - 'lambda:DeleteFunction'
+                  - 'lambda:InvokeFunction'
+                  - 'events:RemoveTargets'
+                  - 'events:DeleteRule'
+                  - 'lambda:RemovePermission'
+                Resource: '*'
+
+  ### cloud watch to kinesis role
+  CloudWatchIAMRole:
+    Type: AWS::IAM::Role
+    Properties:
+      AssumeRolePolicyDocument:
+        Statement:
+          - Effect: Allow
+            Principal:
+              Service: [logs.amazonaws.com]
+            Action: ['sts:AssumeRole']
+      Path: /
+      Policies:
+        - PolicyName: service-autoscaling
+          PolicyDocument:
+            Statement:
+              - Effect: Allow
+                Action:
+                  - 'kinesis:PutRecord'
+                Resource: '*'
+
+  #####################EFS#####################
+  EfsFileStorage:
+    Type: 'AWS::EFS::FileSystem'
+    Properties:
+      BackupPolicy:
+        Status: ENABLED
+      PerformanceMode: maxIO
+      Encrypted: false
+
+      FileSystemPolicy:
+        Version: '2012-10-17'
+        Statement:
+          - Effect: 'Allow'
+            Action:
+              - 'elasticfilesystem:ClientMount'
+              - 'elasticfilesystem:ClientWrite'
+              - 'elasticfilesystem:ClientRootAccess'
+            Principal:
+              AWS: '*'
+
+  MountTargetResource1:
+    Type: AWS::EFS::MountTarget
+    Properties:
+      FileSystemId: !Ref EfsFileStorage
+      SubnetId: !Ref PublicSubnetOne
+      SecurityGroups:
+        - !Ref EFSServerSecurityGroup
+
+  MountTargetResource2:
+    Type: AWS::EFS::MountTarget
+    Properties:
+      FileSystemId: !Ref EfsFileStorage
+      SubnetId: !Ref PublicSubnetTwo
+      SecurityGroups:
+        - !Ref EFSServerSecurityGroup
+
+Outputs:
+  EfsFileStorageId:
+    Description: 'The connection endpoint for the database.'
+    Value: !Ref EfsFileStorage
+    Export:
+      Name: !Sub ${EnvironmentName}:EfsFileStorageId
+  ClusterName:
+    Description: The name of the ECS cluster
+    Value: !Ref 'ECSCluster'
+    Export:
+      Name: !Sub ${EnvironmentName}:ClusterName
+  AutoscalingRole:
+    Description: The ARN of the role used for autoscaling
+    Value: !GetAtt 'AutoscalingRole.Arn'
+    Export:
+      Name: !Sub ${EnvironmentName}:AutoscalingRole
+  ECSRole:
+    Description: The ARN of the ECS role
+    Value: !GetAtt 'ECSRole.Arn'
+    Export:
+      Name: !Sub ${EnvironmentName}:ECSRole
+  ECSTaskExecutionRole:
+    Description: The ARN of the ECS role tsk execution role
+    Value: !GetAtt 'ECSTaskExecutionRole.Arn'
+    Export:
+      Name: !Sub ${EnvironmentName}:ECSTaskExecutionRole
+
+  DeleteCFNLambdaExecutionRole:
+    Description: Lambda execution role for cleaning up cloud formations
+    Value: !GetAtt 'DeleteCFNLambdaExecutionRole.Arn'
+    Export:
+      Name: !Sub ${EnvironmentName}:DeleteCFNLambdaExecutionRole
+
+  CloudWatchIAMRole:
+    Description: The ARN of the CloudWatch role for subscription filter
+    Value: !GetAtt 'CloudWatchIAMRole.Arn'
+    Export:
+      Name: !Sub ${EnvironmentName}:CloudWatchIAMRole
+  VpcId:
+    Description: The ID of the VPC that this stack is deployed in
+    Value: !Ref 'VPC'
+    Export:
+      Name: !Sub ${EnvironmentName}:VpcId
+  PublicSubnetOne:
+    Description: Public subnet one
+    Value: !Ref 'PublicSubnetOne'
+    Export:
+      Name: !Sub ${EnvironmentName}:PublicSubnetOne
+  PublicSubnetTwo:
+    Description: Public subnet two
+    Value: !Ref 'PublicSubnetTwo'
+    Export:
+      Name: !Sub ${EnvironmentName}:PublicSubnetTwo
+
+  ContainerSecurityGroup:
+    Description: A security group used to allow Fargate containers to receive traffic
+    Value: !Ref 'ContainerSecurityGroup'
+    Export:
+      Name: !Sub ${EnvironmentName}:ContainerSecurityGroup
--- a/src/model/cloud-runner/providers/aws/cloud-formations/task-def-formation.yml
+++ b/src/model/cloud-runner/providers/aws/cloud-formations/task-def-formation.yml
@@ -0,0 +1,221 @@
+AWSTemplateFormatVersion: 2010-09-09
+Description: >-
+  AWS Fargate cluster that can span public and private subnets. Supports public
+  facing load balancers, private internal load balancers, and both internal and
+  external service discovery namespaces.
+Parameters:
+  EnvironmentName:
+    Type: String
+    Default: development
+    Description: 'Your deployment environment: DEV, QA , PROD'
+  ServiceName:
+    Type: String
+    Default: example
+    Description: A name for the service
+  ImageUrl:
+    Type: String
+    Default: nginx
+    Description: >-
+      The url of a docker image that contains the application process that will
+      handle the traffic for this service
+  ContainerPort:
+    Type: Number
+    Default: 80
+    Description: What port number the application inside the docker container is binding to
+  ContainerCpu:
+    Type: Number
+    Default: 1024
+    Description: How much CPU to give the container. 1024 is 1 CPU
+  ContainerMemory:
+    Type: Number
+    Default: 2048
+    Description: How much memory in megabytes to give the container
+  BUILDGUID:
+    Type: String
+    Default: ''
+  Command:
+    Type: String
+    Default: 'ls'
+  EntryPoint:
+    Type: String
+    Default: '/bin/sh'
+  WorkingDirectory:
+    Type: String
+    Default: '/efsdata/'
+  Role:
+    Type: String
+    Default: ''
+    Description: >-
+      (Optional) An IAM role to give the service's containers if the code within
+      needs to access other AWS resources like S3 buckets, DynamoDB tables, etc
+  EFSMountDirectory:
+    Type: String
+    Default: '/efsdata'
+  # template secrets p1 - input
+Mappings:
+  SubnetConfig:
+    VPC:
+      CIDR: 10.0.0.0/16
+    PublicOne:
+      CIDR: 10.0.0.0/24
+    PublicTwo:
+      CIDR: 10.0.1.0/24
+Conditions:
+  HasCustomRole: !Not
+    - !Equals
+      - Ref: Role
+      - ''
+Resources:
+  LogGroup:
+    Type: 'AWS::Logs::LogGroup'
+    Properties:
+      LogGroupName: !Ref ServiceName
+    Metadata:
+      'AWS::CloudFormation::Designer':
+        id: aece53ae-b82d-4267-bc16-ed964b05db27
+  SubscriptionFilter:
+    Type: 'AWS::Logs::SubscriptionFilter'
+    Properties:
+      FilterPattern: ''
+      RoleArn:
+        'Fn::ImportValue': !Sub '${EnvironmentName}:CloudWatchIAMRole'
+      LogGroupName: !Ref ServiceName
+      DestinationArn:
+        'Fn::GetAtt':
+          - KinesisStream
+          - Arn
+    Metadata:
+      'AWS::CloudFormation::Designer':
+        id: 7f809e91-9e5d-4678-98c1-c5085956c480
+    DependsOn:
+      - LogGroup
+      - KinesisStream
+  KinesisStream:
+    Type: 'AWS::Kinesis::Stream'
+    Properties:
+      Name: !Ref ServiceName
+      ShardCount: 1
+    Metadata:
+      'AWS::CloudFormation::Designer':
+        id: c6f18447-b879-4696-8873-f981b2cedd2b
+
+  # template secrets p2 - secret
+
+  TaskDefinition:
+    Type: 'AWS::ECS::TaskDefinition'
+    Properties:
+      Family: !Ref ServiceName
+      Cpu: !Ref ContainerCpu
+      Memory: !Ref ContainerMemory
+      NetworkMode: awsvpc
+      Volumes:
+        - Name: efs-data
+          EFSVolumeConfiguration:
+            FilesystemId:
+              'Fn::ImportValue': !Sub '${EnvironmentName}:EfsFileStorageId'
+            TransitEncryption: ENABLED
+      RequiresCompatibilities:
+        - FARGATE
+      ExecutionRoleArn:
+        'Fn::ImportValue': !Sub '${EnvironmentName}:ECSTaskExecutionRole'
+      TaskRoleArn:
+        'Fn::If':
+          - HasCustomRole
+          - !Ref Role
+          - !Ref 'AWS::NoValue'
+      ContainerDefinitions:
+        - Name: !Ref ServiceName
+          Cpu: !Ref ContainerCpu
+          Memory: !Ref ContainerMemory
+          Image: !Ref ImageUrl
+          EntryPoint:
+            Fn::Split:
+              - ','
+              - !Ref EntryPoint
+          Command:
+            Fn::Split:
+              - ','
+              - !Ref Command
+          WorkingDirectory: !Ref WorkingDirectory
+          Environment:
+            - Name: ALLOW_EMPTY_PASSWORD
+              Value: 'yes'
+            # template - env vars
+          MountPoints:
+            - SourceVolume: efs-data
+              ContainerPath: !Ref EFSMountDirectory
+              ReadOnly: false
+          Secrets:
+            # template secrets p3 - container def
+          LogConfiguration:
+            LogDriver: awslogs
+            Options:
+              awslogs-group: !Ref ServiceName
+              awslogs-region: !Ref 'AWS::Region'
+              awslogs-stream-prefix: !Ref ServiceName
+    Metadata:
+      'AWS::CloudFormation::Designer':
+        id: dabb0116-abe0-48a6-a8af-cf9111c879a5
+    DependsOn:
+      - LogGroup
+Metadata:
+  'AWS::CloudFormation::Designer':
+    dabb0116-abe0-48a6-a8af-cf9111c879a5:
+      size:
+        width: 60
+        height: 60
+      position:
+        x: 270
+        'y': 90
+      z: 1
+      embeds: []
+      dependson:
+        - aece53ae-b82d-4267-bc16-ed964b05db27
+    c6f18447-b879-4696-8873-f981b2cedd2b:
+      size:
+        width: 60
+        height: 60
+      position:
+        x: 270
+        'y': 210
+      z: 1
+      embeds: []
+    7f809e91-9e5d-4678-98c1-c5085956c480:
+      size:
+        width: 60
+        height: 60
+      position:
+        x: 60
+        'y': 300
+      z: 1
+      embeds: []
+      dependson:
+        - aece53ae-b82d-4267-bc16-ed964b05db27
+        - c6f18447-b879-4696-8873-f981b2cedd2b
+    aece53ae-b82d-4267-bc16-ed964b05db27:
+      size:
+        width: 150
+        height: 150
+      position:
+        x: 60
+        'y': 90
+      z: 1
+      embeds: []
+    4d2da56c-3643-46b8-aaee-e46e19f95fcc:
+      source:
+        id: 7f809e91-9e5d-4678-98c1-c5085956c480
+      target:
+        id: aece53ae-b82d-4267-bc16-ed964b05db27
+      z: 11
+    14eb957b-f094-4653-93c4-77b2f851953c:
+      source:
+        id: 7f809e91-9e5d-4678-98c1-c5085956c480
+      target:
+        id: c6f18447-b879-4696-8873-f981b2cedd2b
+      z: 12
+    85c57444-e5bb-4230-bc85-e545cd4558f6:
+      source:
+        id: dabb0116-abe0-48a6-a8af-cf9111c879a5
+      target:
+        id: aece53ae-b82d-4267-bc16-ed964b05db27
+      z: 13
--- a/src/model/cloud-runner/providers/aws/cloud-runner-aws-task-def.ts
+++ b/src/model/cloud-runner/providers/aws/cloud-runner-aws-task-def.ts
@@ -0,0 +1,9 @@
+import * as AWS from 'aws-sdk';
+
+class CloudRunnerAWSTaskDef {
+  public taskDefStackName!: string;
+  public taskDefCloudFormation!: string;
+  public taskDefResources: AWS.CloudFormation.StackResources | undefined;
+  public baseResources: AWS.CloudFormation.StackResources | undefined;
+}
+export default CloudRunnerAWSTaskDef;
--- a/src/model/cloud-runner/providers/aws/commands/aws-cli-commands.ts
+++ b/src/model/cloud-runner/providers/aws/commands/aws-cli-commands.ts
@@ -0,0 +1,30 @@
+import AWS from 'aws-sdk';
+import { CliFunction } from '../../../../cli/cli-functions-repository';
+import Input from '../../../../input';
+import CloudRunnerLogger from '../../../services/cloud-runner-logger';
+
+export class AwsCliCommands {
+  @CliFunction(`aws-garbage-collect`, `garbage collect aws`)
+  static async garbageCollectAws() {
+    process.env.AWS_REGION = Input.region;
+    CloudRunnerLogger.log(`Cloud Formation stacks`);
+    const CF = new AWS.CloudFormation();
+    const stacks =
+      (await CF.listStacks().promise()).StackSummaries?.filter((_x) => _x.StackStatus !== 'DELETE_COMPLETE') || [];
+    for (const element of stacks) {
+      CloudRunnerLogger.log(JSON.stringify(element, undefined, 4));
+    }
+    CloudRunnerLogger.log(`ECS Clusters`);
+    const ecs = new AWS.ECS();
+    const clusters = (await ecs.listClusters().promise()).clusterArns || [];
+    if (stacks === undefined) {
+      return;
+    }
+    for (const element of clusters) {
+      const input: AWS.ECS.ListTasksRequest = {
+        cluster: element,
+      };
+      CloudRunnerLogger.log(JSON.stringify(await ecs.listTasks(input).promise(), undefined, 4));
+    }
+  }
+}
--- a/src/model/cloud-runner/providers/aws/index.ts
+++ b/src/model/cloud-runner/providers/aws/index.ts
@@ -0,0 +1,98 @@
+import * as SDK from 'aws-sdk';
+import CloudRunnerSecret from '../../services/cloud-runner-secret';
+import CloudRunnerEnvironmentVariable from '../../services/cloud-runner-environment-variable';
+import CloudRunnerAWSTaskDef from './cloud-runner-aws-task-def';
+import AWSTaskRunner from './aws-task-runner';
+import { ProviderInterface } from '../provider-interface';
+import BuildParameters from '../../../build-parameters';
+import CloudRunnerLogger from '../../services/cloud-runner-logger';
+import { AWSJobStack } from './aws-job-stack';
+import { AWSBaseStack } from './aws-base-stack';
+import { Input } from '../../..';
+
+class AWSBuildEnvironment implements ProviderInterface {
+  private baseStackName: string;
+
+  constructor(buildParameters: BuildParameters) {
+    this.baseStackName = buildParameters.awsBaseStackName;
+  }
+  async cleanup(
+    // eslint-disable-next-line no-unused-vars
+    buildGuid: string,
+    // eslint-disable-next-line no-unused-vars
+    buildParameters: BuildParameters,
+    // eslint-disable-next-line no-unused-vars
+    branchName: string,
+    // eslint-disable-next-line no-unused-vars
+    defaultSecretsArray: { ParameterKey: string; EnvironmentVariable: string; ParameterValue: string }[],
+  ) {}
+  async setup(
+    // eslint-disable-next-line no-unused-vars
+    buildGuid: string,
+    // eslint-disable-next-line no-unused-vars
+    buildParameters: BuildParameters,
+    // eslint-disable-next-line no-unused-vars
+    branchName: string,
+    // eslint-disable-next-line no-unused-vars
+    defaultSecretsArray: { ParameterKey: string; EnvironmentVariable: string; ParameterValue: string }[],
+  ) {}
+
+  async runTask(
+    buildGuid: string,
+    image: string,
+    commands: string,
+    mountdir: string,
+    workingdir: string,
+    environment: CloudRunnerEnvironmentVariable[],
+    secrets: CloudRunnerSecret[],
+  ): Promise<string> {
+    process.env.AWS_REGION = Input.region;
+    const ECS = new SDK.ECS();
+    const CF = new SDK.CloudFormation();
+    CloudRunnerLogger.log(`AWS Region: ${CF.config.region}`);
+    const entrypoint = ['/bin/sh'];
+    const startTimeMs = Date.now();
+
+    await new AWSBaseStack(this.baseStackName).setupBaseStack(CF);
+    const taskDef = await new AWSJobStack(this.baseStackName).setupCloudFormations(
+      CF,
+      buildGuid,
+      image,
+      entrypoint,
+      commands,
+      mountdir,
+      workingdir,
+      secrets,
+    );
+
+    let postRunTaskTimeMs;
+    let output = '';
+    try {
+      const postSetupStacksTimeMs = Date.now();
+      CloudRunnerLogger.log(`Setup job time: ${Math.floor((postSetupStacksTimeMs - startTimeMs) / 1000)}s`);
+      output = await AWSTaskRunner.runTask(taskDef, ECS, CF, environment, buildGuid, commands);
+      postRunTaskTimeMs = Date.now();
+      CloudRunnerLogger.log(`Run job time: ${Math.floor((postRunTaskTimeMs - postSetupStacksTimeMs) / 1000)}s`);
+    } finally {
+      await this.cleanupResources(CF, taskDef);
+      const postCleanupTimeMs = Date.now();
+      if (postRunTaskTimeMs !== undefined)
+        CloudRunnerLogger.log(`Cleanup job time: ${Math.floor((postCleanupTimeMs - postRunTaskTimeMs) / 1000)}s`);
+    }
+    return output;
+  }
+
+  async cleanupResources(CF: SDK.CloudFormation, taskDef: CloudRunnerAWSTaskDef) {
+    CloudRunnerLogger.log('Cleanup starting');
+    await CF.deleteStack({
+      StackName: taskDef.taskDefStackName,
+    }).promise();
+
+    await CF.waitFor('stackDeleteComplete', {
+      StackName: taskDef.taskDefStackName,
+    }).promise();
+    CloudRunnerLogger.log(`Deleted Stack: ${taskDef.taskDefStackName}`);
+    CloudRunnerLogger.log('Cleanup complete');
+  }
+}
+export default AWSBuildEnvironment;