lisc.requester.Requester

class lisc.requester.Requester(wait_time=0.0, logging=None, directory=None)

Object to handle URL requests.

Attributes

is_activebool

Status of the requester, reflecting whether it is currently being used to make requests.

n_requestsint

Number of requests that have been made from this object.

wait_timefloat

Amount of time to wait between requests, in seconds.

start_timestr

Time when request session started.

end_timestr

Time when request session ended.

time_last_reqfloat

Time at which last request was sent.

logging{None, ‘print’, ‘store’, ‘file’}

What kind of logging, if any, to do for requested URLs.

logNone or list or FileObject

Log of requested URLs. Format depends on logging.

__init__(wait_time=0.0, logging=None, directory=None)

Initialize a requester object.

Parameters

wait_timefloat, optional, default: 0.0

Amount of time to wait between requests, in seconds.

logging{None, ‘print’, ‘store’, ‘file’}, optional

What kind of logging, if any, to do for requested URLs.

directorySCDB or str or None, optional

A string or object containing a file path, used for logging.

Examples

Initialize a Requester object, specifying a wait time of 0.1 seconds between requests:

>>> requester = Requester(wait_time=0.1)

Methods

__init__([wait_time, logging, directory])	Initialize a requester object.
as_dict()	Get the attributes of the Requester object as a dictionary.
check()	Print out basic check of requester object.
close()	Set the current object as inactive.
open()	Set the current object as active.
request_url(url)	Request a URL.
set_wait_time(wait_time)	Set the amount of time to rest between requests.
throttle()	Slow down rate of requests by waiting if a new request is initiated too soon.
wait(wait_time)	Pause for specified amount of time.

as_dict()

Get the attributes of the Requester object as a dictionary.

check()

Print out basic check of requester object.

close()

Set the current object as inactive.

open()

Set the current object as active.

request_url(url)

Request a URL.

Parameters

urlstr

Web address to request.

Returns

outrequests.models.Response

Object containing the requested web page.

Examples

Use a Requester object to request the LISC Github repository url:

>>> requester = Requester()
>>> response = requester.request_url('https://github.com/lisc-tools/lisc')

set_wait_time(wait_time)

Set the amount of time to rest between requests.

Parameters

wait_timefloat

Time, in seconds, to wait between launching URL requests.

Examples

Set the wait time to 0.1 seconds:

>>> requester = Requester()
>>> requester.set_wait_time(0.1)

throttle()

Slow down rate of requests by waiting if a new request is initiated too soon.

static wait(wait_time)

Pause for specified amount of time.

Parameters

wait_timefloat

Time to wait between launching URL requests, in seconds.

Examples using lisc.requester.Requester

URLs and Requests

URLs and Requests