Social networking is currently one of the hottest things on the Web, so it comes as no surprise that many web applications are trying to incorporate social networking content to attract more customers. The challenge that many developers are experiencing is that, despite the attempts to standardize the APIs[1], the majority of the existing social network websites are still providing either proprietary APIs or support different versions of OpenSocial. As a result, there is currently a proliferation of the APIs and a wealth of open source code for supporting them[2]. In this article I will discuss how an OpenSocial implementation, Apache Shindig[3] can be used to alleviate some of these issues.
I will start with discussing OpenSocial standards and Shindig architecture and then show how they can be used for bringing social networking content to an application. As an example I will describe usage of Shindig for OpenSocial enablement of Gypsii social network[4] implementation.
OpenSocial APIs
As defined by the OpenSocial Specification[5]:
“OpenSocial is a set of APIs for building social applications that run on the web. OpenSocial's goal is to make more apps available to more users, by providing a common API that can be used in many different contexts. Developers can create applications, using standard JavaScript and HTML, that run on social websites that have implemented the OpenSocial APIs. These websites, known as OpenSocial containers, allow developers to access their social information; in return they receive a large suite of applications for their users.
The OpenSocial APIs expose methods for accessing information about people, their friends, and their data, within the context of a [social network] container.”
Social applications revolve around people and their activities and relationships. People are a fundamental part of social networking software and the OpenSocial API. The Person object provides access to a user's information that, depending on the site, can include anything from "favorite TV shows" to "my work aspirations", to “favorite places”. The OpenSocial API provide support for viewing and changing personal data. The other important user information is the set of connections they have in the social graph. Being able to share information and interact with friends changes the dynamic of user experience. OpenSocial supports the notion of user’s friends and provides APIs for retrieving person’s friends and their information. When an application wants to interact with or display data for "friends of friends", the OpenSocial specification supports extending a query for friends by adding an optional filter - this filter allows you to, for example, sort friends by their distance from a user. Containers can optionally support "friends of friends" queries, "friends of friends of friends" queries, and so on. OpenSocial also allows friends to share information about the things they have been doing lately - these are referred to as activities. OpenSocial exposes activity streams, which are a collection of actions a user has taken. Activity templates allow application developers to define messages with placeholders for pieces of application or user data. This separation of data and presentation allows multiple activities to be combined into activity summaries: consolidated bundles of activities that let users know what their friends are up to without having to wade through a flood of messages. Another important aspect of social interaction is the ability to read, post, and delete messages between users in the network. OpenSocial defines a number of message types including public messages (such as profile comments) and private messages (messages restricted to certain individuals and groups). Messages are grouped into MessageCollections that include a unique id, name and sizing on the number of messages.
The current specification defines two versions of the APIs - JavaScript[6] and REST[7].
The OpenSocial JavaScript specification is design to support OpenSocial functionality within a Gadget[8]. A typical architecture for this case is presented in the Figure 1
Figure 1 Typical architecture for using JavaScript OpenSocial APIs
In this case a user application is built as a collection of Gadgets, where a Gadget itself uses the OpenSocial APIs for accessing social information. As a result, the JavaScript APIs are tightly coupled with the Gadget APIs and might not always be applicable for not Gadget-based implementation.
The OpenSocial REST API, on the other hand, is not directly connected with Gadgets and allows any client applications to view and publish users' OpenSocial data. We will use these APIs for our implementation.
Apache Shindig
Apache Shindig is an open source implementation of both Gadget APIs and OpenSocial APIs. Below is a brief recap of what a Shindig is; Apache Shindig is an OpenSocial container and helps you start hosting OpenSocial apps quickly by providing the code to render gadgets, proxy requests, and handle REST and RPC requests.
Figure 2 Apache Shindig[9]
When it comes to OpenSocial, Shindig implements both JavaScript and REST APIs. The overall architecture of Shindig OpenSocial implementation[10] is presented in the Figure 3.
Figure 3 Overall Architecture of Shindig Open Social implementation
As we can see from Figure 3 the Shindig implementation includes two major parts - the client JavaScript container, and Shindig server. The server is a WAR file, which can run in any servlet container - we were using Tomcat. As defined above, for our implementation we are using REST OpenSocial APIs directly and consequently we are not going to discuss the client JavaScript container here.
The server implementation contains two listeners - JSON RPC servlet and data service (REST) servlet. Each servlet is supported by its own handler. Consequently there are two handlers - RPC and REST handler. Both handlers share the same infrastructure including date converters (not shown here) and a set of OpenSocial handlers :
- People handler - a handler supporting user management operations such as retrieving information about users and their relationships
- Activity handler - a handler supporting retrieving and storing activity information related to a user
- Data handler - a handler supporting storing and retrieving application data. Application data is an item of data, such as a preference setting, stored by the social network on a user's behalf for a given application
- Message handler - a handler used for sending messages and retrieving messages to/from users of the social network
Shindig server implementation is based on Dependency Injection using Guice framework[11]. As a result it is using a binding class (configured in application’s web.xml file), which defines services, invoked by each of the handlers - every handler is using a corresponding service. This architecture makes it extremely simple to overwrite the service implementations which are provided as part of Shindig distribution with custom ones, such as one which implements an integration with existing social network applications.
Gypsii
Gypsii[12] is a location-aware social network. In addition to traditional user attribution, common for social network implementations, Gypsii also allows its participants to store and share place and location information, including address, geo coordinates, optional pictures and descriptions. A special place in Gypsii is a current user location. It also supports POIs (Points Of Interest) and advertisement.
Gypsii provides a set of APIs - GyPSii OpenExperience API(OEx)[13] - XML-RPC APIs, providing access to all of the Gypsii functionality. The GyPSii variant of XML-RPC is inspired by the XML-RPC specification[14], but has been tailored to the needs of GyPSii. The protocol defines a client-server communication pattern where both sides use XML to communicate with the other. The Gypsii site for developers provides documentation on both the XML-RPC protocol and implementation of Java clients using it
Implementation
Overall Implementation Architecture
The overall architecture of our implementation is presented at Figure 4. We are leveraging existing Shindig support for OpenSocial REST support, but replacing the default sample OpenSocial container provided by Shindig with the custom provider, implemented using an adapter to Gypsii. Please note that this custom provider could connect to any other social network provider, such as Facebook, Myspace or Ning.
Figure 4 Overall Solution architecture
The advantage of this approach is that we are effectively shielding our application from specifics of social network implementation/integration approach. Client APIs are provided by Shindig and are controlled internally. As a result our applications do not depend on specific technology and APIs exposed by social network provider. These details are encapsulated by an adapter for specific network.
Security
One of the complex issues with integration of multiple social network providers is security/credential support, ranging from support for a login using name/password to OpenID[15], to OAuth[16]. Our implementation is based on a simple identity conversion schema presented in Figure 5.
Figure 5 Identity conversion
User request Shindig with NAVTEQ’s user ID - token. An adapter for a given social network converts this token into a credential, required by a network. This conversion is based on the user store, which contains the information required for this conversion, whether it is username/password, OpenID, specific key or anything else required for a specific conversion.
This allows us to use NAVTEQ’s user ID as a standardized path parameter for OpenSocial REST APIs. It also allow for a simple implementation of routing of the request between multiple social networks providers, based on the following naming convention:
OpenSocialUID = NAVTEQUID@Network
For example user ID 123468@gypsii means accessing Gypsii for NAVTEQ user with ID 123468. This allows for building of a very simple network chooser (Figure 4), which routes request to the appropriate social network adapter based on the provided username.
Overwriting of Default Social Service
Due to the support of dependency injection, overwriting of the default social network service for Shindig is fairly straightforward and involves the following main steps:
- Implementing custom OpenSocial service. Custom service can be implemented either as a single class (Listing 1) or as a set of classes, collectively implementing APIs defined by Shindig.
public class CustomOpenSocialService implements ActivityService,
AppDataService, PersonService, MessageService {
…………………………………..
}
Listing 1 Custom OpenSocial service
- Creating binding class. Once the custom OpenSocial service is in place (we are assuming a single one - Listing 1), a binding class has to be implemented (Listing 2) telling Shindig to invoke it for processing requests.
public class NAVTEQBind extends SocialApiGuiceModule {
………………
bind(ActivityService.class).to(CustomOpenSocialService.class);
bind(AppDataService.class).to(CustomOpenSocialService.class);
bind(PersonService.class).to(CustomOpenSocialService.class);
bind(MessageService.class).to(CustomOpenSocialService.class);
………………
}
Listing 2 Binding Custom OpenSocial service
- Configuring Shindig to use binding class. Finally, this binding has to be made known to Shindig implementation. This is done through a web.xml configuration change (Listing 3). This configuration is processed by the
org.apache.shindig.common.servlet.GuiceServletContextListenerclass which is invoked on startup of the Shindig application and sets the binding correctly.
<context-param>
<param-name>guice-modules</param-name> <param-value>
org.apache.shindig.common.PropertiesModule:com.navteq.opensocial.bind.NAVTEQBind </param-value>
</context-param>
Listing 3 Web.config changes
Accessing OpenSocial REST APIs from the browser
We need to access our OpenSocial REST APIs from the browser. The issue here is the same-origin policy limitations[17] imposed by modern browsers:
“The same-origin policy prevents a script loaded from one domain from getting or manipulating properties of a document from another domain. That is, the domain of the requested URL must be the same as the domain of the current Web page. This basically means that the browser isolates content from different origins to guard them against manipulation.”
There are several common approaches for overcoming this limitation:
- Have the Web page GET/POST data from/to the Web server page originates from, and have the web server behave as a proxy relaying the request to the actual third-party servers. Although widely used, this technique isn't scalable and adds latency (which grows in the case of large paylods) to request processing.
- Use frame elements to create new areas in the current Web page for accessing any third-party content. This often works well for POSTs with no return data, but is rarely usable in the case of GET due to the limited access to data between frames. Also, after being fetched, the content in the frames would be subject to the same-origin policy limitations.
- Implement GET returning JSONP (JSON with Padding) - JSON data wrapped in a function call. In this case the data is returned in the script. When the script loads, it executes. It works because the same-origin policy doesn't prevent dynamic script insertions and treats the scripts as if they were loaded from the domain that provided the Web page.
The OpenSocial APIs provided by Shindig are leveraging both GET for fetching information from the social network and POST for updating information. For support of GET we are leveraging JSONP, while for POST we use the framing technique.
For implementing JSONP support on the server we used an open source JSONP filter[18], which does not require any programmatic changes to Shindig. Usage of this filter is defined in Shindig’s web.xml (Listing 4)
<filter>
<display-name>jsonp</display-name>
<filter-name>jsonp</filter-name>
<filter-class>org.jsonp.JsonpFilter</filter-class>
<init-param>
<param-name>jsonp</param-name>
<param-value>jsonpCallback</param-value>
</init-param>
<init-param>
<param-name>json-mime-types</param-name>
<param-value>application/json</param-value>
</init-param>
</filter>
<filter-mapping>
<filter-name>jsonp</filter-name>
<url-pattern>*</url-pattern>
</filter-mapping>
Listing 4 Configuration for JSONP filter
We also had to slightly modify Shindig code to be able to support posting from a frame. The Shindig distribution disallows content type "application/x-www-form-urlencoded" because it interferes with OAuth body signing. Because we are not using OAuth and "application/x-www-form-urlencoded" is the content type used by the browser POST, we had to modify class org.apache.shindig.protocol.ContentTypes (Listing 5)and org.apache.shindig.protocol.DataServiceServlet (Listing 6) to allow "application/x-www-form-urlencoded".
………………….
public static void checkContentTypes(Set<String> allowedContentTypes,
String contentType, boolean disallowUnknownContentTypes) throws InvalidContentTypeException {
if (StringUtils.isEmpty(contentType)) {
if (disallowUnknownContentTypes) {
throw new InvalidContentTypeException(
"No Content-Type specified. One of "
+ StringUtils.join(allowedContentTypes, ", ") + " is required");
} else {
// No content type specified, we can fail in other ways later.
return;
}
}
contentType = ContentTypes.extractMimePart(contentType);
//BL. comented out to support
// if (ContentTypes.FORBIDDEN_CONTENT_TYPES.contains(contentType)) {
// throw new InvalidContentTypeException(
// "Cannot use disallowed Content-Type " + contentType);
// }
..........................................
Listing 5 Modification to ContentTypes class
……………
public static final Set<String> ALLOWED_CONTENT_TYPES =
new ImmutableSet.Builder<String>().addAll(ContentTypes.ALLOWED_JSON_CONTENT_TYPES)
.addAll(ContentTypes.ALLOWED_XML_CONTENT_TYPES)
.addAll(ContentTypes.FORBIDDEN_CONTENT_TYPES)
.addAll(ContentTypes.ALLOWED_ATOM_CONTENT_TYPES).build();
……………………
Listing 6 Modification to DataServiceServlet class
Another issue is that browser POST submits name/value pairs. Although, Shindig supports name/value pairs for POST, it also supports processing data straight from the POST body. An implementation adds body content as an additional name/value parameter. The issue is that in the case of browser submission, a body is an empty body and content of the parameter gets overwritten. A change to the org.apache.shindig.protocol.DefaultHandlerRegistry class (Listing 7) fixes this problem
……………..
public Future<?> execute(Map<String, String[]> parameters, Reader body,
SecurityToken token, BeanConverter converter) {
try {
// bind the body contents if available
if (body != null) {
String bString = IOUtils.toString(body);
if(bString.length() > 0)
parameters.put(operation.bodyParam(), new String[]{bString});
}
RequestItem item = methodCaller.getRestRequestItem(parameters, token, converter,
beanJsonConverter);
listener.executing(item);
return methodCaller.call(handlerProvider.get(), item);
………………………………………..
Listing 7 Disabling empty body overwrite
Finally, when POST does not return any data, Shindig’s implementation converts empty data into an empty JSON object, which is returned with the content type "application/json". This causes the browser to try to initiate dialog with the user. A change org.apache.shindig.protocol.DataServiceServlet (Listing 7) solves this problem:
org.apache.shindig.protocol.DataServiceServlet (Listing 7) solves this problem:
servletResponse.setContentType(converter.getContentType());
if (responseItem.getErrorCode() >= 200 && responseItem.getErrorCode() < 400) {
Object response = responseItem.getResponse();
// TODO: ugliness resulting from not using RestfulItem
if (!(response instanceof DataCollection) && !(response instanceof RestfulCollection)) {
//BL - modified to not return fake responce
if(response instanceof Map){
if(((Map)response).isEmpty()){
servletResponse.setContentType("text/plain");
return;
}
}
response = ImmutableMap.of("entry", response);
}
Listing 8 Disabling empty response
Sample application
An OpenSocial implementation described in this article was used for building the GeoSpatial OpenSocial mashup, allowing the placing of a user and his friends and/or their favorite places on the map (Figure 6)
Figure 6 User and his friends on the map
It also allows route calculation to either user or/and his favorite places and use of social network messaging (Figure 7) to submit them to a friend
Figure 7 Social network messaging
This simple implementation is just one of many potential synergies between social networking and geospatial data. Many other interesting mashups can be implemented, based on the described approach for combining the two.
Conclusion
Proliferation of the open social APIs makes it difficult to build applications supporting multiple existing social networks. A solution, proposed in this article, allows to overcome some of these challenges by “hiding” these differences from front-end implementation through an introduction of an intermediary providing standards-based APIs. The choice of Apache Shindig, providing such APIs and a robust “pluggable” architecture, significantly simplified overall implementation.
Acknowledgements
I am thankful to my NAVTEQ colleagues, especially Stefan Balkowiec and Robert Camp for many discussions on implementation architecture and Andreas Schuetten for pointing me to JSONP and implementing the GUI support.
[1] http://www.opensocial.org/
[2] http://www.thegoodnamesweregone.com/post/83-open-source-web-api-frameworks.aspx
[3] http://incubator.apache.org/shindig/
[5] http://www.opensocial.org/Technical-Resources/opensocial-spec-v09/OpenSocial-Specification.html
[6] http://wiki.opensocial.org/index.php?title=JavaScript_API_Reference
[7] http://wiki.opensocial.org/index.php?title=OpenSocial_REST_Developer%27s_Guide
[8] http://www.opensocial.org/Technical-Resources/opensocial-spec-v09/Gadgets-API-Specification.html
[9] Source : http://incubator.apache.org/shindig/
[10] Rajdeep Dua. Architectural Overview of Shindig , an OpenSocial Reference Implementation. http://chrisschalk.com/shindig_docs/rajdeep/shindig-overview/onjava-shindig-overview-tidy.html and Rajdeep Dua. Overview of REST Implementation in Shindig - Java Version. http://sites.google.com/site/opensocialarticles/Home/shindig-rest-java
[11] http://code.google.com/p/google-guice/
[12] http://www.gypsii.com/home.cgi
[13] http://developer.gypsii.com/docs.cgi?op=view&id=api.html
[17] Seda Özses, Salih Ergül Cross-domain communications with JSONP, Part 1: Combine JSONP and jQuery to quickly build powerful mashups. http://www.ibm.com/developerworks/library/wa-aj-jsonp1/