Post request using multipart/form-data for uploading file not working in python but working fine in cURL

General
1

I am trying to create bulkOperationRunMutation for updating shopify products https://shopify.dev/docs/api/usage/bulk-operations/imports#generate-the-uploaded-url-and-parameters

for this, I am trying to upload the data to the google storage bucket, so that shopify and fetch the data from that bucket and run the bulk operation query. In cURL it is working fine but then send the post request using python it is giving me error InvalidArgumentInvalid argument.Cannot create buckets using a POST.

Python code

url = "https://url_to_bucket/"
payload={
    'Content-Type': 'text/jsonl',
    'success_action_status': '201',
    'acl': 'private',
    'key': 'upload_path_on_the_bucket',
    'x-goog-date': '20230404T073135Z',
    'x-goog-credential': 'merchant_credentials',
    'x-goog-algorithm': 'GOOG4-RSA-SHA256',
    'x-goog-signature': 'signature_token',
    'policy': 'policy_token',
    'file': 'file_contant'
}
headers = {
  'Content-Type': 'multipart/form-data'
}
response = requests.request("POST", url, headers=headers, data=payload)

cURL Code

curl --location 'https://url_to_bucket/' \
--header 'Content-Type: multipart/form-data' \
--form 'Content-Type="text/jsonl"' \
--form 'success_action_status="201"' \
--form 'acl="private"' \
--form 'key="upload_path_on_the_bucket"' \
--form 'x-goog-date="20230404T073135Z"' \
--form 'x-goog-credential="merchant_credentials"' \
--form 'x-goog-algorithm="GOOG4-RSA-SHA256"' \
--form 'x-goog-signature="signature_token"' \
--form 'policy="policy_token"' \
--form 'file="file_contant"'

Any help will be appreciated. Thank you

Tried documentation of shopify and python to make a multipart/form-data request but nothing worked for me

2

To successfully upload data to a Google Cloud Storage bucket using Python, you’ll need to make sure your multipart/form-data request is structured correctly. The issue you are encountering could be due to how you’re forming the request in your Python code.

Here’s how you can adapt your Python code to match the structure used in your cURL command:

Updated Python Code

Make sure to use the requests library’s built-in support for multipart form data:

import requests

url = "https://url_to_bucket/"

# Prepare the payload with the required fields
payload = {
    'Content-Type': 'text/jsonl',
    'success_action_status': '201',
    'acl': 'private',
    'key': 'upload_path_on_the_bucket',
    'x-goog-date': '20230404T073135Z',
    'x-goog-credential': 'merchant_credentials',
    'x-goog-algorithm': 'GOOG4-RSA-SHA256',
    'x-goog-signature': 'signature_token',
    'policy': 'policy_token',
}

# Prepare the file content
file_content = 'your_file_content_here'  # Replace with actual file content

# Create a dictionary for the file to upload
files = {
    'file': ('filename.jsonl', file_content, 'text/jsonl')  # Replace 'filename.jsonl' with your desired filename
}

# Make the POST request
response = requests.post(url, headers={'Content-Type': 'multipart/form-data'}, data=payload, files=files)

# Check the response
if response.status_code == 200:
    print("Upload successful!")
else:
    print(f"Error: {response.status_code} - {response.text}")

Key Changes

  1. File Upload:
  • Instead of including the file content directly in the payload, use the files argument of requests.post(). This allows requests to handle the multipart form-data encoding correctly.
  1. Filename:
  • Make sure to specify a filename when adding the file to the files dictionary. This simulates the file being uploaded, which is essential for the request.
  1. Header Content-Type:
  • While requests automatically sets the Content-Type for multipart requests, if you’re manually setting headers, you don’t need to specify Content-Type as multipart/form-data directly. You can simply omit the headers argument since requests will handle it automatically.

Troubleshooting

  • Ensure that the URL to the bucket is correct and that you have permission to upload files to that bucket.
  • Verify that the credentials and tokens used in the payload are accurate and not expired.
  • You can log the response from the server to understand if there are more specific error messages to guide you.

This should help you correctly structure your multipart/form-data request to upload files to Google Cloud Storage using Python! Let me know if you have any further questions.