Skip to content

Twitter extractor will retrieve "/home" instead of a tweet URL #520

Description

@Divi

Twitter is now a fully logged-on website: you cannot access a tweet without an account.
So, the extractor will try to request the twitter.com/xxx/status/xxx but will follow location on /home (with the login screen) and will attempt to call the oembed API with /home URI.

The only fix that I found to disable this behavior is to disable the "follow redirection" behavior on cURL.

$client = new CurlClient();
$client->setSettings([
    'follow_location' => false
]);

$embed = new Embed(new Crawler($client));

We may use the cookie to inject the auth_token cookie, but I'm not sure the token won't change after a few hours/days.

This may impact other embeds, so if you have a better solution, please let me know!

No activity

Activity on this issue will appear here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions