Amazon S3 manifest

This is a sample manifest, showcasing the structure with references to documents stored in Amazon S3 and associated metadata for each entry.

1. Manifest file with source_ref and metadata

[
  {
    "source_ref": "s3://examplebucket/blogs/example.com-How to Optimize AWS Lambda for Scalability.pdf",
    "metadata": {
      "user_name": "Alice Cooper"
    }
  },
  {
    "source_ref": "s3://examplebucket/blogs/example.com-Introduction to Amazon S3 Glacier for Data Archiving.pdf",
    "metadata": {
      "user_name": "Bob Johnson"
    }
  },
  {
    "source_ref": "s3://examplebucket/blogs/example.com-AWS EC2 Instance Types and Choosing the Right One.pdf",
    "metadata": {
      "user_name": "Charlie Daniels"
    }
  },
  {
    "source_ref": "s3://examplebucket/blogs/example.com-Amazon DynamoDB: Best Practices and Common Pitfalls.pdf",
    "metadata": {
      "user_name": "Diana Smith"
    }
  },
  {
    "source_ref": "s3://examplebucket/blogs/example.com-Deep Dive into Amazon RDS Performance Tuning.pdf",
    "metadata": {
      "user_name": "Edward White"
    }
  },
  {
    "source_ref": "s3://examplebucket/blogs/example.com-Amazon ElasticSearch Service: What You Need to Know.pdf",
    "metadata": {
      "user_name": "Fiona Green"
    }
  },
  {
    "source_ref": "s3://examplebucket/blogs/example.com-Optimizing CloudWatch Monitoring for AWS Environments.pdf",
    "metadata": {
      "user_name": "George Harris"
    }
  }
]

• source_ref: A reference to the file stored in an S3 bucket, typically a URL or path pointing to the file.

• metadata: An object containing metadata information, such as the user's name who uploaded the file or is associated with it.

  • user_name: The name of the user associated with this particular document or data entry.

1. Manifest file with sorce_ref, processed_source_ref and metadata

[
  {
    "source_ref": "s3://examplebucket/resumes/JohnDoe_Resume.pdf",
    "processed_source_ref": "s3://examplebucket/resumes/jsonlfiles/JohnDoe_Resume_123456.json",
    "metadata": {
      "name": "John Doe",
      "experience": "6y_2m",
      "source": "cybersecurity-analyst-resumes"
    }
  },
  {
    "source_ref": "s3://examplebucket/resumes/JaneSmith_Resume.pdf",
    "processed_source_ref": "s3://examplebucket/resumes/jsonlfiles/JaneSmith_Resume_654321.json",
    "metadata": {
      "name": "Jane Smith",
      "experience": "4y_8m",
      "source": "cybersecurity-analyst-resumes"
    }
  },
  {
    "source_ref": "s3://examplebucket/resumes/MichaelBrown_Resume.pdf",
    "processed_source_ref": "s3://examplebucket/resumes/jsonlfiles/MichaelBrown_Resume_789123.json",
    "metadata": {
      "name": "Michael Brown",
      "experience": "7y_5m",
      "source": "cybersecurity-analyst-resumes"
    }
  },
  {
    "source_ref": "s3://examplebucket/resumes/SusanWhite_Resume.pdf",
    "processed_source_ref": "s3://examplebucket/resumes/jsonlfiles/SusanWhite_Resume_321654.json",
    "metadata": {
      "name": "Susan White",
      "experience": "9y_3m",
      "source": "cybersecurity-analyst-resumes"
    }
  },
  {
    "source_ref": "s3://examplebucket/resumes/PaulGreen_Resume.pdf",
    "processed_source_ref": "s3://examplebucket/resumes/jsonlfiles/PaulGreen_Resume_987654.json",
    "metadata": {
      "name": "Paul Green",
      "experience": "5y_0m",
      "source": "cybersecurity-analyst-resumes"
    }
  }
]

• source_ref: The S3 path or URL to the original unprocessed document.

• processed_source_ref: The S3 path or URL to the processed version of the document (usually in JSON format).

• metadata: Contains additional information about the document such as the user's name, experience, and the source of the document.

  • name: The name of the person associated with the document.

  • experience: The professional experience of the person, typically in "years_months" format.

  • source: The category or source of the document (e.g., "cybersecurity-analyst-resumes").