Jump to content









Photo

Carriots and PythonAnywhere

Python

  • Please log in to reply
11 replies to this topic

#1 Zambiot

Zambiot

    Advanced Member

  • Members
  • PipPipPip
  • 94 posts
  • LocationAsia

Posted 29 March 2013 - 01:48 AM

I wanted to point out that PythonAnywhere can be used to connect and work with Carriots.  I've asked PythonAnywhere to whitelist api.carriots.com on their free account, so that all can access Carriots and they did!  This means you can write Python scripts or use a Python console to interact with Carriots.  Furthermore you can write your front-end app or web interface in Python and have Carriots be your backend.

 

Perhaps in this thread we can post various scripts for using Carriots API with Python.  I'll get things started.  Add your own code for others to use.

 

In my streams I have a bunch of bogus test data that I wanted to clean out.  Sure, it probably isn't hurting anything by being there, however, I don't need it and it helps to test and work with known good information and a clean slate.  The following bit of Python code uses Requests to get a List of streams from Carriots and then Delete each stream.  BE CAREFUL RUNNING THIS CODE - IT WILL DELETE!  

 

[EDIT]  A double warning, this code will delete both Status Streams and Data Streams unless you specify a parameter to narrow down the list.  See the API and the use of ?_t=sta.[/EDIT]

 

If you want to filter the list from Carriots, look at List in the API and then add elements in the params section.  For instance  you can filter the list that is retrieved from Carriots for a specific device.  Just add your device's name and comment the params line.

 

Be sure to insert your API Key.

 

This code is using Python 2.7 and uses Requests and JSON.

import requests, json

app_url = "http://api.carriots.com/"
app_element = "streams/"
app_filter = ""
app_key = "Your_API_Key_Here"

headers = {"carriots.apikey":app_key}

#Enter any filter parameters in the params section below
params = {}
#params = {'device': 'testDevice@carriots'}

print ""

print "Getting streams from Carriots..."
carriotsResponse = requests.get(app_url + app_element, params=params, headers=headers)
binary = carriotsResponse.content
output = json.loads(binary)

#Print all streams retrieved
#print output

print "There are " + str(len(output['result'])) + " streams retrieved from Carriots."

print "Deleting each stream..."
good = 0
fail = 0

for stream in range(len(output['result'])):
    carriotsResponse = requests.delete(app_url + app_element + output['result'][stream]['id_developer'] + "/", headers=headers)

    if carriotsResponse.status_code == requests.codes.ok:
        good += 1
    else:
        fail += 1

print ""

print "Total streams processed: good=" + str(good) + " fail=" + str(fail)
print ""


#2 alvaro

alvaro

    Carriots CTO

  • Administrators
  • 72 posts

Posted 29 March 2013 - 07:53 AM

Great news! I think it will be a good start point to test Carriots API and share useful scripts.
Thanks again.

Alvaro Everlet

Carriots CTO


#3 Zambiot

Zambiot

    Advanced Member

  • Members
  • PipPipPip
  • 94 posts
  • LocationAsia

Posted 01 April 2013 - 02:17 PM

Here is another piece of Python code for use with Carriots.  This code is not too different from above, but what this code does is list all items, up to 1000, that match the requested parameters.  Uncomment your desired parameters and edit the values.  Add in any other parameters you want that you see in the API Streams List method.  This particular script has the output suppressed, but that is easily changed if you like to see lots of data scrolling by in front of your eyes.  

 

import requests, json

app_url = "http://api.carriots.com/"
app_element = "streams/"
app_filter = ""
app_key = "YOUR_API_KEY_HERE"

headers = {"carriots.apikey":app_key}

#Enter any filter parameters in the params section below
params = {}
#params = {'device':'testdevice@Carriots'}
#params = {'device':'testdevice@Carriots', 'at_from':1364468250, 'at_to':1364774400}

print ""

print "Getting streams from Carriots..."
carriotsResponse = requests.get(app_url + app_element, params=params, headers=headers)
binary = carriotsResponse.content
output = json.loads(binary)

#Print all streams retrieved
#print output

print ""
print "There are " + str(len(output['result'])) + " streams retrieved from Carriots."
print "The URL of the request: " + carriotsResponse.url
print ""

 

 



#4 Zambiot

Zambiot

    Advanced Member

  • Members
  • PipPipPip
  • 94 posts
  • LocationAsia

Posted 03 April 2013 - 01:40 PM

How about another Python script?  This takes the user's input of Device, Start Date, End Date and timezone offset and then will print out the number of streams for each hour in the provided range for the specified device.

 

I created this script so that I could look at the historical performance of a device's success in logging data to Carriots.

 

# This program takes a user entry for device, a start date and an end date.
# Then the program will then list stream count for each hour.

import requests, json
import datetime as dt

# Kick off the program by getting the user input and echo back
print ""
print "This program will show the total number of streams for each hour for the specified device."
print ""
deviceRI = raw_input("Please enter the name of the device in form test@Carriots: ").strip()
dateStartRI = raw_input("Please enter the date of the first day of data in form YYYY-MM-DD: ").strip()
dateEndRI = raw_input("Please enter the date of the last day of data in form YYYY-MM-DD: ").strip()
timezoneRI = raw_input("Please input your timezone offset.  For example, -5 or +8 or 0 (UTC): ").strip()

print ""
print "Data will be listed for " + deviceRI + " from " + dateStartRI + " to " + dateEndRI

# Calculate the time timestamps.
epochStart = dt.datetime(1970,1,1)

dateStartRI = dt.datetime.strptime(dateStartRI, '%Y-%m-%d')
timestampStart = int((dateStartRI - epochStart).total_seconds() - (int(timezoneRI) * 60 * 60))
#print timestampStart

dateEndRI = dt.datetime.strptime(dateEndRI, '%Y-%m-%d')
timestampEnd = int((dateEndRI - epochStart).total_seconds() - (int(timezoneRI) * 60 * 60) + (((23 * 60) + 59) * 60) + 59)
#print timestampEnd

# Build the request
app_url = "http://api.carriots.com/"
app_element = "streams/"
app_key = "YOUR_API_KEY_HERE"

headers = {"carriots.apikey":app_key}

#Loop through each hour, getting data from Carriots and show how many streams there are
print ""
for hr in xrange(timestampStart, timestampEnd, 3600):
    at_from = hr
    at_to = hr + 3600

    params = {'device':deviceRI, 'at_from':at_from, 'at_to':at_to}

    carriotsResponse = requests.get(app_url + app_element, params=params, headers=headers)
    binary = carriotsResponse.content
    output = json.loads(binary)

    print (dt.datetime.fromtimestamp(at_from + (int(timezoneRI) * 60 * 60)).strftime('%Y-%m-%d Hour: %H')) + " has " + str(len(output['result'])) + " streams."

print ""
print "end"
print ""

 

 



#5 Zambiot

Zambiot

    Advanced Member

  • Members
  • PipPipPip
  • 94 posts
  • LocationAsia

Posted 13 April 2013 - 01:36 AM

I've got a new script for you today.  The script below will generate a CSV output for your streams based on the device name and date range input.  In order to use the script, you will need to first update it with your Carriots API Key, as well as input the names of your data in two places.  The first place is where the CSV builds a header row, the other place is where the CSV has rows added through each stream iteration.  

 

I suspect this script could be more Pythonic as well as abstracted so you don't have to input your data names.  I've tested this script on Python 2.7 using PythonAnywhere.

 

If you have improvements to this script, a Python script of your own to share, or have a script request, please let me know through PM or a post on this thread.

 

# This program takes a user entry for device, a start date and an end date.
# Then the program will get lists of streams from Carriots and arrange them into
# a csv file.

import requests, json, csv
import datetime as dt


# Kick off the program by getting the user input and echo back
print ""
print "This program will build CSVs with data from Carriots."
print ""
deviceRI = raw_input("Please enter the name of the device in form test@Carriots: ").strip()
dateStartRI = raw_input("Please enter the date of the first day of data in form YYYY-MM-DD: ").strip()
dateEndRI = raw_input("Please enter the date of the last day of data in form YYYY-MM-DD: ").strip()
timezoneRI = raw_input("Please input your timezone offset such as -5 or +8: ").strip()

print ""
print "Data will be gathered for " + deviceRI + " from " + dateStartRI + " to " + dateEndRI

#Build the file name
fileName = dateStartRI + " " + deviceRI + ".csv"

# Calculate the time timestamps.
epochStart = dt.datetime(1970,1,1)

dateStartRI = dt.datetime.strptime(dateStartRI, '%Y-%m-%d')
timestampStart = int((dateStartRI - epochStart).total_seconds() - (int(timezoneRI) * 60 * 60))
#print timestampStart

dateEndRI = dt.datetime.strptime(dateEndRI, '%Y-%m-%d')
timestampEnd = int((dateEndRI - epochStart).total_seconds() - (int(timezoneRI) * 60 * 60) + (((23 * 60) + 59) * 60) + 59)
#print timestampEnd

#Prepare a CSV file for writing to.  Change and expand "data1" and "data2" to match the data names in your stream
myCSV = csv.writer(open(fileName, "wb+"))
myCSV.writerow(["id_developer", "created_at", "at", "protocol", "owner", "device",
                "data1", "data2"])
totalRowsWritten = 1

# For building the request
app_url = "http://api.carriots.com/"
app_element = "streams/"
app_key = "YOUR_API_KEY_HERE"

headers = {"carriots.apikey":app_key}

#Loop through each hour and show how many streams there are (getting data from Carriots) and put into CSV
print ""
for hr in xrange(timestampStart, timestampEnd, 3600):
    at_from = hr
    at_to = hr + 3600

    #Adjust parameters for sorting or display as you would like
    params = {'device':deviceRI, 'at_from':at_from, 'at_to':at_to, 'sort':'at'}
    #params = {'device':deviceRI, 'created_at_from':at_from, 'created_at_to':at_to, 'sort':'created_at'}

    carriotsResponse = requests.get(app_url + app_element, params=params, headers=headers)
    binary = carriotsResponse.content
    output = json.loads(binary)

    #Again, change and expand 'data1' and 'data2' to match your data names
    for stream in range(len(output['result'])):
        myCSV.writerow([output['result'][stream]['id_developer'],
                        output['result'][stream]['created_at'],
                        output['result'][stream]['at'],
                        output['result'][stream]['protocol'],
                        output['result'][stream]['owner'],
                        output['result'][stream]['device'],
                        output['result'][stream]['data']['data1'],
                        output['result'][stream]['data']['data2']])

        totalRowsWritten += 1

    print (dt.datetime.fromtimestamp(at_from + (int(timezoneRI) * 60 * 60)).strftime('%Y-%m-%d Hour: %H')) + " has " + str(len(output['result'])) + " streams."

print ""
print "There were " + str(totalRowsWritten) + " rows created in the CSV"
print "end"
print ""

 

 

 



#6 mcastillo

mcastillo

    Carriots CEO

  • Administrators
  • 34 posts
  • LocationMadrid

Posted 14 April 2013 - 11:25 AM

For those of you who want to learn Python or want to improve you level: Tomorrow (April 15th) starts on Coursera a new course on Python. More info here https://www.coursera...teractivepython

And thanks Zambiot for your Scripts anf for pointing out PythonAnyware as a great frontend platform. Can't wait to see your final product! :)

Regards to all,
Miguel

#7 Zambiot

Zambiot

    Advanced Member

  • Members
  • PipPipPip
  • 94 posts
  • LocationAsia

Posted 21 April 2013 - 03:03 AM

I personally like what I saw for Python learning on http://www.codecademy.com.

 

I'll try and cook-up more scripts in May when my development focuses more to the online portion of my design.  



#8 Zambiot

Zambiot

    Advanced Member

  • Members
  • PipPipPip
  • 94 posts
  • LocationAsia

Posted 25 April 2013 - 03:31 AM

Here is another Python script that can be used to test the Carriots API for the Now function.  This particular script is more educational than a utility.  The value is seeing how the parameters work in a URL to produce the desired date/time format.  You could use HURL.it or cURL, but I prefer to practice in Python on PythonAnywhere.

 

# This script is for checking the time as given by Carriots
# See here for the API information:  https://www.carriots.com/documentation/api/dates

import requests, json

app_url = "http://api.carriots.com/"
app_element = "now/"
app_filter = ""
app_key = "YOUR_API_KEY_HERE"

headers = {"carriots.apikey":app_key}

#Enter any parameters in the params section below or use one of the examples provided
params = {}                        #defaults of date in ISO8601 based at UTC
#params = {'type':'timestamp'}      #returns the unix based timestamp, format and time_zone are ignored
#params = {'type':'date', 'format':'Y-m-d'}             #returns just the date, no time
#params = {'type':'date', 'time_zone':'US/Eastern'}     #returns the full ISO8601 date/time for USA eastern timezone

print ""

print "Getting the date/time from Carriots..."
carriotsResponse = requests.get(app_url + app_element, params=params, headers=headers)
binary = carriotsResponse.content
output = json.loads(binary)

#Print all streams retrieved
print output
print "The URL used: " + carriotsResponse.url

print ""

 

-Zambiot



#9 jpastor

jpastor

    Development leader

  • Administrators
  • 159 posts

Posted 26 April 2013 - 03:24 PM

Great job Zambiot, thanks again!

Enjoy Carriots



#10 alvaro

alvaro

    Carriots CTO

  • Administrators
  • 72 posts

Posted 29 April 2013 - 07:17 AM

Thank for your contributions!


Alvaro Everlet

Carriots CTO


#11 sifane

sifane

    Newbie

  • Members
  • Pip
  • 1 posts

Posted 14 November 2015 - 01:01 AM

Hi !

 

How to write a 'POST' request in python please ?

 

Thanks



#12 syntechzob

syntechzob

    Member

  • Members
  • PipPip
  • 13 posts
  • LocationCape Town

Posted 14 November 2015 - 02:11 PM

Hi

 

Personally I use unirest http://unirest.io/python.html.

 

Steven







Also tagged with one or more of these keywords: Python

3 user(s) are reading this topic

0 members, 3 guests, 0 anonymous users