Showing posts with label useragent. Show all posts
Showing posts with label useragent. Show all posts

Sunday, August 7, 2011

Perl 101: Accessing HTTPS URL via LWP

LWP(Library for WWW in Perl) is a Perl library that you can use to access the URL via back-end. However, accessing the URL via HTTPS needs SSL verification which stops your script from running.

Basically, if you are to access the HTTPS URL via browser, it will prompt you to verify the certificate which you have to accept before you actually be able to access it successfully.

To do it via back-end, we will have to do the same but this time, no interface.

First, we need to do SSL verification on the actual server that you will be using to access the URL, and to do that, we will use LYNX command and save the certificate if prompted.

Please see sample syntax below.

lynx https://example.mydomain.com/myscript.php..

If you don't have LYNX, you can do the normal way by accessing it via the browser. All you need to do is to access the browser of the actual server and save the certificate from there.

To check if successful, try to access the URL again, and you should not be prompted to save the certificate again. Hence, saving certificate is not successful.

Once okay with the certificate, we are now good to use LWP library to access the URL.

Please take note that the libraries below should be installed in your server.
- LWP::UserAgent
- HTTP::Request
- Crypt::SSLeay

The script below will be able to access the HTTPS URL. Hope you like it!

#!/usr/bin/perl
use LWP;

my $url = 'https://example.mydomain.com/myscript.php';
my $ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 0 });
$ua->timeout(60000);

my $res = $ua->get($url);
print "result: $res\n";

1;

Sunday, March 27, 2011

How to retrieve your twitter feeds in PERL

Retrieving twitter feeds is very simple and easy, you only need an HTTP request that returns format such as JSON, XML, RSS, and ATOM. This method doesn't require authentication.

Just follow the simple steps below using PERL. Hope you like it!

1. Install LWP::UserAgent in your server.

2. Initialize libraries, header and user agent.

require LWP::UserAgent;

my $lwpua = LWP::UserAgent->new;

my $uagent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6";
my @header = ('Referer' => 'http://api.twitter.com/',
             'User-Agent' => $uagent);

3. Compose the HTTP request.

my $twuser = '<your twitter username>';
my $twurl = "http://api.twitter.com/1/statuses/user_timeline.<format>?screen_name=$twuser";

Supported Format:
  • JSON
  • XML
  • RSS
  • ATOM
URL Parameters:
  • user_id - The id of twitter account you want to retrieve.
  • screen_name - The username of twitter account you want to retrieve.
  • since_id - This will return the feeds after or more than the specified message id.
  • max_id - This will return the feeds before or less than the specified message id.
  • count - The number of tweets to be retrieve.
  • page - This will specify the page of the result to retrieve.
  • trim_user - When set to either true, t or 1, each tweet returned in a timeline will include a user object including only the status authors numerical id.
  • include_rts - When set to either true, t or 1, the timeline will contain native retweets (if they exist) in addition to the standard stream of tweets.
  • include_entities - When set to either true, t or 1, each tweet will include a node called "entities". This node offers a variety of metadata about the tweet in a discreet structure, including: user_mentions, urls, and hashtags. While entities are opt-in on timelines at present, they will be made a default component of output in the future.
4. Execute HTTP GET request.

my $response = $lwpua->get($twurl, @header);
my $return = $response->content;


That's it!! Please see below for the complete code using JSON format.

#!/usr/bin/perl

require LWP::UserAgent;
use JSON;

my $lwpua = LWP::UserAgent->new;

my $uagent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6";
my @header = ('Referer' => 'http://api.twitter.com/', 'User-Agent' => $uagent);

my $twuser = '<your twitter username>';
my $twurl = "http://api.twitter.com/1/statuses/user_timeline.json?screen_name=$twuser";

my $response = $lwpua->get($twurl, @header);
my $return = $response->content;

my $json = JSON->new->allow_nonref;
my $json_text = $json->decode($return);

my @tweets = @{$json_text};

my $message;
foreach $tweet (@tweets)
{
  $message .= $tweet->{text}."\n";
}

print "$message";



1;

Resource: dev.twitter.com

Friday, February 11, 2011

Login to twitter via backend using PERL

This is the same from the last post - Login to foursquare via backend using PERL but this time, we're doing it for twitter. Just follow the 4 simple steps below.

1. Install the libraries into your server.
  • LWP::UserAgent;
  • HTTP::Cookies;
2. Initialize the libraries UserAgent and Cookies.

my $lwpua = LWP::UserAgent->new;

my $user_agent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6";
my @header = ('Referer' => 'http://m.twitter.com',
             'User-Agent' => $user_agent);

my $cookie_file = "cookies.dat";
my $cookie_jar = HTTP::Cookies->new(
                file => $cookie_file,
                autosave => 1,
                ignore_discard => 1);

$lwpua->cookie_jar($cookie_jar);

3. Post your username and password to https://mobile.twitter.com/session and save the cookies.

my $strUser = '<your twitter username>';
my $strPass = '<your twitter password>';

# log-in to twitter
$response = $lwpua->post('https://mobile.twitter.com/session',
                            ['username' => $strUser,
                             'password' => $strPass,
                             'authenticity_token' => $auth_token], @header);

$cookie_jar->extract_cookies( $response );
$cookie_jar->save;

4. You can check if successful or not by accessing https://mobile.twitter.com via GET and print the content.

$response = $lwpua->get('https://mobile.twitter.com', @header);
$form_data = $response->content;

print "$form_data\n\ndone!";


That's it! Please see the complete code below.

#!/usr/bin/perl

require LWP::UserAgent;

use strict;
use warnings;

use HTTP::Cookies;

my $lwpua = LWP::UserAgent->new;

my $user_agent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6";
my @header = ('Referer' => 'http://m.twitter.com',
             'User-Agent' => $user_agent);

my $cookie_file = "cookies.dat";
my $cookie_jar = HTTP::Cookies->new(
                file => $cookie_file,
                autosave => 1,
                ignore_discard => 1);

$lwpua->cookie_jar($cookie_jar);

# getting authenticity token
my $response = $lwpua->get('https://mobile.twitter.com/session/new', @header);
my $form_data = $response->content;

$form_data =~ s/\n//g;
$form_data =~ /input name="authenticity_token" type="hidden" value="(.*?)"/ig;

my $auth_token = $1;

my $strUser = '<your twitter username>';
my $strPass = '<your twitter password>';

# log-in to twitter
$response = $lwpua->post('https://mobile.twitter.com/session',
                            ['username' => $strUser,
                             'password' => $strPass,
                             'authenticity_token' => $auth_token], @header);

$cookie_jar->extract_cookies( $response );
$cookie_jar->save;

$response = $lwpua->get('https://mobile.twitter.com', @header);
$form_data = $response->content;

print "$form_data\n\ndone!";


1;


Hope you like it!! Thank you for reading.

Thursday, February 10, 2011

Login to foursquare via backend using PERL

This is just a simple login request to foursquare via backend using PERL. Just follow the simple steps and the complete code below.

1. Install the libraries needed.
  • LWP::UserAgent;
  • HTTP::Cookies;
2. Initialize UserAgent and Cookies.

my $lwpua = LWP::UserAgent->new;

my $user_agent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6";
my @header = ('Referer' => 'http://foursquare.com',
             'User-Agent' => $user_agent);

my $cookie_file = "cookies.dat";
my $cookie_jar = HTTP::Cookies->new(
                file => $cookie_file,
                autosave => 1,
                ignore_discard => 1);

$lwpua->cookie_jar($cookie_jar);

3. Post your username and password to https://foursquare.com/mobile/login and save the cookies.

my $strUser = "<your foursquare username>";
my $strPass = "<your foursquare password>";

# log-in to foursquare
my $response = $lwpua->post('https://foursquare.com/mobile/login',
                      ['username' => $strUser,
                       'password' => $strPass], @header);
$cookie_jar->extract_cookies( $response );
$cookie_jar->save;

4. You can check if successful or not by accessing http://foursquare.com/mobile/ via GET and print the content.

$response = $lwpua->get("http://foursquare.com/mobile/", @header);
my $form_data = $response->content;

print "$form_data\n\ndone!";


That's it! So simple right? Please see the complete code below.

#!/usr/bin/perl

require LWP::UserAgent;

use strict;
use warnings;

use HTTP::Cookies;

my $lwpua = LWP::UserAgent->new;

my $user_agent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6";
my @header = ('Referer' => 'http://foursquare.com',
             'User-Agent' => $user_agent);

my $cookie_file = "cookies.dat";
my $cookie_jar = HTTP::Cookies->new(
                file => $cookie_file,
                autosave => 1,
                ignore_discard => 1);

$lwpua->cookie_jar($cookie_jar);

my $strUser = "<your foursquare username>";
my $strPass = "<your foursquare password>";

# log-in to foursquare
my $response = $lwpua->post('https://foursquare.com/mobile/login',
                      ['username' => $strUser,
                       'password' => $strPass], @header);
$cookie_jar->extract_cookies( $response );
$cookie_jar->save;

$response = $lwpua->get("http://foursquare.com/mobile/", @header);
my $form_data = $response->content;

print "$form_data\n\ndone!";


1;


Hope you like it!! Thank you for reading. Please see other post - Login to twitter via backend using PERL

Thursday, December 23, 2010

How to post facebook status via backend in PERL

We know that facebook site is a secured and very strict with regards to their website and other site components. There is so much happening on the site that users doesn't realize it changes constantly and that is also for security reason, why? that is because hackers do some research, checking the site constantly as well and if your site is not changing, they can come up with a plan and boom!! before you know it, you're doom already.

Enough with the applaud to the company coz they were great already.

I'm not here to lecture about hacking, but how to submit facebook status right? so, just follow the steps below and you can have a BOT that will send facebook status without human intervention.

1. First thing you have to do is to know the structure of the wap site, this time the facebook wap site http://m.facebook.com

2. Since we are using PERL, we have to install some libraries for sending HTTP request and cookies. install the libraries below:
  • LWP::UserAgent
  • HTTP::Cookies

3. After installing the 2 libraries into your server, we are now ready to do the coding. The code below is just a standard coding that we do in PERL.

#!/usr/bin/perl

require LWP::UserAgent;

use strict;
use warnings;

use HTTP::Cookies;


4. After we require the libraries, we need to initialize the variables we need.

my $lwpua = LWP::UserAgent->new;

5. Set your facebook username/email, password, and your status. please see below.

my $strUser = "<your facebook username/email>";
my $strPass = "<your facebook password>";
my $strStatus = "<your facebook status>";


6. Set up user agent that we will use to send HTTP request.

my $user_agent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6";
my @header = ( 'Referer' => 'http://m.facebook.com/','User-Agent' => $user_agent);


7. Set up cookie file and jar.

my $cookie_file = "cookies.dat";
my $cookie_jar = HTTP::Cookies->new(
                file => $cookie_file,
                autosave => 1,
                ignore_discard => 1);

$lwpua->cookie_jar($cookie_jar);


8. Setup the code for log-in to facebook. Please take note that you should know the link of the login page of the site which is http://m.facebook.com/login.php. The important thing here is to look for the FORM element in which the site is submitting the credentials. You also have to get all the important INPUT parameters for submission to the FORM. I pasted here the FORM element of the login page source as of today as reference. Below also are the ONLY details you need to login.

=begin
<form method="post" action="https://login.facebook.com/login.php?m=m&amp;refsrc=http%3A%2F%2Fm.facebook.com%2login.php&amp;fbb=rac4881b1&amp;refid=9">
<input class="input" name="email" value="" type="text"/>
<input class="input" name="pass" type="password"/>
<input type="submit" value="Log In" class="btn btnC" name="login"/>
</form>
=cut

# logging in to facebook
my $response = $lwpua->post('http://m.facebook.com/login.php',
                      ['email' => $strUser,
                       'pass' => $strPass,
                       'login' => 'Login'], @header);


9. After logging in, we have to extract the COOKIES and use it on our next SESSION or PAGE.

$cookie_jar->extract_cookies( $response );
$cookie_jar->save;


10. After saving the COOKIES, accessing the facebook home page should direct you to the home page where in you can send your STATUS. You can check that if you print the $response->content. I also pasted the code snippet of the content below as reference.

$response = $lwpua->get('http://m.facebook.com/home.php', @header);
my $form_data = $response->content;

=begin
<form method="post" id="composer_form" action="/a/home.php?fbb=r488fb708&amp;refid=7"><input type="hidden" name="fb_dtsg" value="DdtLy" autocomplete="off" /><input type="hidden" name="post_form_id" value="726fdf3c15403500a70f64960565e305" /><input type="hidden" name="charset_test" value="€,´,€,´,水,Д,Є" /><div>What&#039;s on your mind?<br /><textarea class="composerInput" name="status" rows="2" onfocus="" onblur=""></textarea></div><input type="submit" value="Share" class="btn btnC" name="update" /></form>
=cut


11. Same thing here with the login page but this time, we need to know how the page sends a status message. find the FORM that submits STATUS. please take note that the site is frequently changing so you should know how to do the REGEX to catch all the PARAMETERS you need and for the benefit of this post, the above content is the updated content as of today and my REGEX below should be able to catch parameters such as: FORM ACTION, FB_DTSG, and POST_FORM_ID.

$form_data =~ s/\n//g;
$form_data =~ /form method="post" id="composer_form" action="(.*?)"(.*?)name="fb_dtsg" value="(.*?)"(.*?)name="post_form_id" value="(.*?)"/ig;

my $form_action = $1;
my $form_fbdtsg = $3;
my $form_id = $5;


12. Send now the STATUS message with the PARAMETERS needed below.

@header = ('Referer' => 'http://m.facebook.com/',
           'User-Agent' => $user_agent);

$response = $lwpua->post('http://m.facebook.com'.$form_action,
                           ['fb_dtsg' => $form_fbdtsg,
                            'post_form_id' => $form_id,
                            'status' => $strStatus,
                            'update' => 'Share'], @header);

$form_data = $response->content;
unlink($cookie_file);

1;


13. And, that's it!! below is the whole script. hope you were able to get it. next will be in PHP. yeah men!!

#!/usr/bin/perl

require LWP::UserAgent;

use strict;
use warnings;

use HTTP::Cookies;

my $lwpua = LWP::UserAgent->new;

my $strUser = "<your facebook username>";
my $strPass = "<your facebook password>";
my $strStatus = "<your facebook status>";

my $user_agent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6";
my @header = ( 'Referer' => 'http://m.facebook.com/','User-Agent' => $user_agent);

my $cookie_file = "cookies.dat";
my $cookie_jar = HTTP::Cookies->new(
                file => $cookie_file,
                autosave => 1,
                ignore_discard => 1);

$lwpua->cookie_jar($cookie_jar);

my $response = $lwpua->post('http://m.facebook.com/login.php',
                      ['email' => $strUser,
                       'pass' => $strPass,
                       'login' => 'Login'], @header);
$cookie_jar->extract_cookies( $response );
$cookie_jar->save;
$response = $lwpua->get('http://m.facebook.com/home.php', @header);

my $form_data = $response->content;

$form_data =~ s/\n//g;
$form_data =~ /form method="post" id="composer_form" action="(.*?)"(.*?)name="fb_dtsg" value="(.*?)"(.*?)name="post_form_id" value="(.*?)"/ig;

my $form_action = $1;
my $form_fbdtsg = $3;
my $form_id = $5;

@header = ('Referer' => 'http://m.facebook.com/',
           'User-Agent' => $user_agent);

$response = $lwpua->post('http://m.facebook.com'.$form_action,
                           ['fb_dtsg' => $form_fbdtsg,
                            'post_form_id' => $form_id,
                            'status' => $strStatus,
                            'update' => 'Share'], @header);

$form_data = $response->content;
unlink($cookie_file);

1;

Leadership 101


  • Leadership demands sacrifices for the near-term to receive lasting benefits. the longer we wait to make sacrifices, the harder they become. Successful people make important decisions early in their life, then manage those decisions the rest of their lives.
  • Growth does not happen by chance. If you want to be sure to grow, you need a plan something strategic, specific, and scheduled. it's a discipline that would need incredible determination from us.
  • Success comes by going the extra mile, working the extra hours, and investing the extra time. The same is true for us. If we want to get to excel in any segment of life, a little extra effort can help. Our efforts can go a long way if we only work a little smarter, listen a little better, push a little harder, and persevere a little longer.
  • Making a difference in your work is not about productivity; it's about people. When you focus on others and connect with them, you can work together to accomplish great things.
  • Envision a goal you'd like to reach. Make it big enough to scare you a little. Now write down a plan for moving toward it. Create mini-goals within the big goal, to set yourself up for continual progress. And include some risks, too. Set yourself up for success.
  • Leaders build margins, not image. A leader may be forced to take unpopular stands for the good of the company. Popularity isn't bad, but decisions made solely on the basis of popular opinion can be devastating. So take courage and make the right though sometimes painful choices.