아주가끔은제정신

Prolog

You probably know my last year's SCN Blog post "A story about Twitter, XML and WD4A" ;-)
In April this year I've received a short DM from Mark F. "getting #SAP #ABAP on this list would be wholesome" (he referted to the site: http://dev.twitter.com/pages/libraries).
Nice idea, I thought, give me time until the summer break.

End of July I wanted to refresh my Twitter API knowledge by reading the docs and I saw this message on the dev site: "The @twitterapi team will be shutting off basic authentication on the Twitter API. All applications, by this date, need to switch to using OAuth."
No problem, with OAuth I've battled already while developing Wave and Streamwork apps. (haha, more later)

Chapter 1 - The Twitter API

The Twitter REST(?) API is pretty good described on http://dev.twitter.com/doc I think, no further explanation is needed here.

Chapter 2 -  The JSON Parser

In my Twitter WDA client I've used the XML response. Since I felt in love with JSON while working with Python, I've decided to use JSON this time.
First prob: how to parse JSON ABAP?
My search on SCN found the nice JSON function group from Quentin Dubois (Wiki page)
Because the Twitter response contains not only flat data but also embedded objects (a status object always contains a user object) and some responses are arrays, the mentioned function group is not really the solution I needed, so I've decided to write my own parser (but with parts of the code of the module, hope Quentin doesn't kill me now).

The result of a parsed JSON object is now a hashed key/value table, where we can read each element by simply call e.g. "text = simplejson->get_value( 'text' ).".
Is the result of the read element again an object, you have just to parse it again:
user = simplejson->get_value( 'user' ).       "returns another object
user_data = simplejson->parse_object( user ). "parse object
simplejson->set_data( user_data ).            "set new data in parser
screen_name = simplejson->get_value( 'screen_name' ).  "get element

The result of a parsed JSON array is a standard table of the hashed key/value table.

With this tiny simplejson helper class I wrote my first twitter API classes, and since the basic authentification is not cut off yet, all test went well until here.

Chapter 3 - OAuth

The next step of the journey was the implementation of OAuth. With a look at my Python sources (Streamwork OAuth implementation) and the first chapters of the Twitter OAuth docs it seemed very familiar and I began with the realization.


An OAuth request contains, amongst others, two fields called "oauth_nonce", a string with random characters, and "oauth_timestamp", the seconds counted from Jan. 1st 1970.
Because there are no standard functions (I think so), I've developed two small helper methods:

method CREATE_NONCE. CONSTANTS: chars TYPE string VALUE '0123456789ABCDEFGHIJKLMNOPQRSTUVWXTZabcdefghiklmnopqrstuvwxyz'. DATA: rnd TYPE i. DO length TIMES. "input parameter (twitter needs 8 chars) CALL FUNCTION 'QF05_RANDOM_INTEGER' EXPORTING ran_int_max = 60 ran_int_min = 0 IMPORTING ran_int = rnd. CONCATENATE nonce chars+rnd(1) INTO nonce. ENDDO. endmethod. method CREATE_TIMESTAMP. CONSTANTS: unix TYPE d VALUE '19700101'. "Unix' birthday DATA: timestamp_i TYPE i , diff TYPE i . GET TIME. diff = sy-tzone. "diff to UTC in secs IF sy-dayst = 'X'. "daylight saving active? ADD 3600 TO diff. ENDIF. timestamp_i = ( sy-datum - unix ) * 86400 "days in secs + sy-uzeit(2) * 3600 "hours in secs + sy-uzeit+2(2) * 60 "mins in secs + sy-uzeit+4(2) "secs - diff "diff to UTC in secs . timestamp = timestamp_i. timestamp = timestamp(10). "w/o sign (trailing space) endmethod.
Hint under friends: if the timestamp is not correct, Twitter will refuse your request, believe me ;) -> set your system time correctly!

-> read more: OAuth at Twitter

Chapter 4 - HMAC-SHA1

But the first test results brought me back down to earth: I've overseen the tiny remark "Twitter requires that all OAuth requests be signed using the HMAC-SHA1 algorithm." WTF?
Streamwork uses PLAINTEXT authentification, but what is HMAC-SHA1? Googlegooglegoogle

The search brought me two results (ok, much more than two, but these two are the most relevant ones):
- a SHA1-Javascript library
- a simple note in an SCN-Forum post, means there is a function module called "CALCULATE_HASH_FOR_CHAR"

An SHA1 function module? Great. Looking into the source of FM "CALCULATE_HASH_FOR_CHAR" and the question marks in my brain appeared again (only a system-call in it). What does the FM docu say? Nothing, no docu available. The usage of this FM was definitely too "hot" for me. What, if I overwrote some needed cryptographic stuff in the system. Not fatal on my own systems, but what about client systems? No, thanks.
Fortunately I remembered, that I've read somewhere somewhat about the usage of Javascript within ABAP. SE24, "CL_*JAVA*", <F4> -> et voilá: found the class "CL_JAVA_SCRIPT".
Google again -> points me to this SAP help site

Again WTF....

But thanks god (and SAP!), we still have the old docus available: here you can find the relevant part from NW7.0

Chapter 5 - Javascript

Although I don't like Javascript very much, playing around with the CL_JAVA_SCRIPT class, I was surprised about the functionality of the class. Even whole ABAP-OO classes can be bound the Javascript source.
A CL_PYTHON would definitely be better, but the class works great atm and is probably the only way to use open source libraries for functions not delivered by SAP!

Back to topic:
my first experiments with the class I've done like described in the docu: with inline code. But for sure, this is not the solution I want to build into the API. Where to store the Javascript sources? Where they belong: in the Mime repository.
Now we have the SHA1 library and an additional single liner called twibap.js stored in the mime repository and with this code snipped we can load the source back into an ABAP string:
*--- load Javascript sources from mime repository ---* mime_api = cl_mime_repository_api=>get_api( ). mime_url = '/SAP/PUBLIC/BC/ztwibap/sha1.js'. mime_api->get( EXPORTING i_url = mime_url IMPORTING e_content = mime_content ). CALL FUNCTION 'ECATT_CONV_XSTRING_TO_STRING' EXPORTING im_xstring = mime_content IMPORTING ex_string = sha1_source. mime_url = '/SAP/PUBLIC/BC/ztwibap/twibap.js'. mime_api->get( EXPORTING i_url = mime_url IMPORTING e_content = mime_content ). CALL FUNCTION 'ECATT_CONV_XSTRING_TO_STRING' EXPORTING im_xstring = mime_content IMPORTING ex_string = twibap_source. CONCATENATE sha1_source twibap_source INTO js_source SEPARATED BY cl_abap_char_utilities=>cr_lf.
twibap.js contains:
abap.oauth_signature = b64_hmac_sha1(abap.oauth_secret, abap.basestring) + '=';
and with this code we finally can sign the message:
*--- compile source and bind variables ---* js_processor = cl_java_script=>create( ). js_processor->bind( EXPORTING name_obj = 'abap' name_prop = 'oauth_secret' CHANGING data = me->oauth_secret ). basestring = create_basestring( method ). js_processor->bind( EXPORTING name_obj = 'abap' name_prop = 'basestring' CHANGING data = basestring ). js_processor->bind( EXPORTING name_obj = 'abap' name_prop = 'oauth_signature' CHANGING data = me->oauth_signature ). return_value = js_processor->evaluate( js_source ).
In addition I only had to develop my own encoding method called “percent_encode”, because the "cl_http_utility=>escape_url()" method doesn't fit to the OAuth dictate, where the only characters you can ignore are "- _ . ~" (and some other abnormalities).

The whole Twitter workflow works nice now, but I was not very satisfied with this JS solution.
Therefore back to SAP notes and google for a deeper search for more information about the function module "CALCULATE_HASH_FOR_CHAR".

Chapter 6 - The SecureStore

In Note 1416202 I finally found the answer. The function modules are NOT "secret", but "The raw documentation was not activated."
With NW 7.01 SP7 the documentation will be delivered (I was so close to install SP7 on my system..., but luckily I found the docu in the infinite vastness of the internet).

In the documentation of the function group "SECH" and its function modules we can read, that we can use these function modules for our own purposes. So did I:
"*--- set secret to SecureStorage ---* CALL FUNCTION 'SET_HMAC_KEY' EXPORTING keycstr = me->oauth_secret client_independent = space. "*--- calculate base64 signature ---* CALL FUNCTION 'CALCULATE_HMAC_FOR_CHAR' EXPORTING alg = 'SHA1' data = basestring key_must_exist = 'X' IMPORTING hmacbase64 = signature.
Hey, it works! Party! Trashed the Javascript part.

Boom, Dump, failed again. What happened? In the first steps of the OAuth authorization process (request_token etc.) the oauth_secret contains only the consumer_secret (42 characters + "&").
The function module 'SET_HMAC_KEY' works brilliant until that point, where I want to sign a user action (e.g. sending a tweet). In this case the secret combines the consumer secret and the token secret (of the user). The function module responses with an "parameter_length" exception.
With some experiments I found out, that the FM only accepts 81 characters as maximum. Hey, why? I only want to SET the key, no process at this moment. And in addition: nowhere in the HMAC-SHA1 OAuth key definition is a length maximum mentioned.

In my despair I opened a SCN forum thread.
And what a surprise (or not): no 24 hours later I've got the solution :) SCN members rock!

The solution: if the key is longer than 81 characters, we have to store the hash of the key, not the key itself (still don’t know why).
The code snippet:
"*--- FM 'SET_HMAC_KEY' doesn't accept keys > 81 chars ---* IF STRLEN( me->oauth_secret ) < 82. "*--- set secret to SecureStorage ---* CALL FUNCTION 'SET_HMAC_KEY' EXPORTING keycstr = me->oauth_secret client_independent = space. ELSE. "*--- hash the secret ---* CALL FUNCTION 'CALCULATE_HASH_FOR_CHAR' EXPORTING data = me->oauth_secret IMPORTING hashx = hashx. secret_hashed = hashx. "*--- set hashed secret to SecureStorage ---* CALL FUNCTION 'SET_HMAC_KEY' EXPORTING keyxstr = secret_hashed client_independent = space. ENDIF.
Now the Twitter API is finally finished...

Chapter 7 - A simple client

... and we can build our first simple (very simple!) Twitter client (output of our own home timeline):
DATA: twitter_api TYPE REF TO zcl_twa_api , statuses TYPE zcl_twa_status_t , text TYPE string , id TYPE string , user TYPE REF TO zcl_twa_user , screen_name TYPE string . FIELD-SYMBOLS: <status> TYPE REF TO zcl_twa_status. twitter_api = zcl_twa_api=>getapi( screen_name = 'your_twitter_name' ). statuses = twitter_api->timeline_res->home_timeline( ). LOOP AT statuses ASSIGNING <status>. text = <status>->gettext( ). id = <status>->getid( ). user = <status>->getuser( ). screen_name = user->getscreenname( ). WRITE:/ id, screen_name, 25 text. ENDLOOP.
Time to build a SAPLINK nuggets for the beta test. To test the nugget I've imported it to another system, activated all sources (ignored errors, because of recursive definitions) and started the test client.
Oh noooo, again -> Dump: FM 'SET_HMAC_KEY' does not exist.

Chapter 8 - Back to Chapter 5

Why? And why me? The answer of the first question is easy. I've developed the API on 7.01 and imported the nugget in a 7.00 system (but I cannot answer the second Q)
Thanks to my decades long experience ;) I've only stared out the Javascript part. Now I only had to activate the part again, create a nugget especially for 7.00 systems and include the Mime objects into the nugget again.

Epilog

You: "And for what is it good for?"
Me: "No idea"
You: "But why did you make it"
Me: "You could also ask: Why are you running Android 2.2 (Froyo) on a WindowsMobile phone. The answer would be the same: Because it works, and it makes so much fun ;)"


(probably the tiniest Froyo phone of the world)

http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/20474%3Futm_source%3Dtwitterfeed%26utm_medium%3Dtwitter%26utm_campaign%3DFeed%253A+SAPNetworkWeblogs+%2528SAP+Network+Weblogs%2529