Rectangle 27 0

Python Scrapy override content type to be multipartform data on post Request?


import scrapy
from scrapy.http import FormRequest

class MySpider(scrapy.Spider):
    # ...
    def start_requests(self):
        yield FormRequest(some_post_url,
                          formdata=dict(param1='value1', param2='value2'))
yield FormRequest(..., headers={'Content-Type': 'multipart/form-data'})

Just use scrapy.http.FormRequest instead of scrapy.Request, passing the parameters in the formdata argument.

This is what we are using, it encodes our form data using "application/x-www-form-urlencoded", adding in percent symbols when we want "multipart/form-data" which doesn't interfere with our data (carriage returns and newline byte-codes). Is there a way to specify this?

Note