Take Data Sync for a Spin

Last year, we implemented a big chunk of the JPA 2.0 API on the client side. In many cases, this allows you to use your existing JPA entities on the client and on the server, simply by putting them in a shared package which is visible to the GWT compiler and deployed to the server.

This is great, because it allows one model class to fill many roles:

1. Define the persistence structure on the server side (via Hibernate or any other JPA provider)
2. Define the persistence structure on the client side (via ErraiJPA)
3. Act as the backing model for forms on the client side (via Errai UI and Data Binding)
4. Act as the validation specification to all of the above (via Bean Validation annotations)

Saving all this duplication of structure, persistence logic, form modeling, and validation rules is great for many reasons, but I think the most important reason of all is that is makes the application easy to maintain and adapt to changing requirements. You remain nimble because the structure of the problem you're solving is defined in one (and only one!) place.

Enter Data Sync

But what's all this about data sync in the blog title, then?

Well, now you can add one more thing to the list of roles your shared model classes fill: They define data sets that can be kept in sync between the client and the server!

Here's an example of how easy it is to use the new JPA Data Sync feature:

First, let's say these are our shared model classes that are already doing double duty on the client and the server:

@Portable @Bindable @Entity

@NamedQueries({

  @NamedQuery(name="currentItemsForUser", query="SELECT i FROM TodoItem i WHERE i.user = :user AND i.archived=false ORDER BY i.text"),

  @NamedQuery(name="allItemsForUser", query="SELECT i FROM TodoItem i WHERE i.user = :user ORDER BY i.text")

})

public class TodoItem {

  @Id @GeneratedValue

  private Long id;

  /**

   * The user who owns this To-do item.

   */

  @ManyToOne(cascade=CascadeType.MERGE)

  private User user;

  private String text;

  private Boolean done = Boolean.FALSE;

  private Boolean archived = Boolean.FALSE;

  public Long getId() {

    return id;

  }

  public void setId(Long id) {

    this.id = id;

  }

  public User getUser() {

    return user;

  }

  public void setUser(User user) {

    this.user = user;

  }

  public String getText() {

    return text;

  }

  public void setText(String text) {

    this.text = text;

  }

  public Boolean isDone() {

    return done;

  }

  public void setDone(Boolean done) {

    this.done = done;

  }

  public Boolean isArchived() {

    return archived;

  }

  public void setArchived(Boolean archived) {

    this.archived = archived;

  }

  @Override

  public String toString() {

    return "TodoItem [id=" + id + ", user=" + (user == null ? "null" : user.getId()) + ", done=" + done +

            ", archived=" + archived + ", text=" + text + "]";

  }

}

@Portable @Bindable @Entity @Table(name="todolist_user")

@NamedQueries({

  @NamedQuery(name="userById", query="SELECT u FROM User u WHERE u.id = :userId"),

  @NamedQuery(name="userByEmail", query="SELECT u FROM User u WHERE u.email = :email")

})

public class User {

  @Id @GeneratedValue

  private Long id;

  /**

   * The name the user wants us to call them, both to themselves and other users.

   */

  @NotNull

  @Size(min=1, max=60)

  private String shortName;

  /**

   * The user's full name.

   */

  @NotNull

  @Size(min=1, max=60, message="Is that really your name? I'd like to meet your parents.")

  private String fullName;

  /**

   * The user's email address.

   */

  @Column(nullable=false, unique=true)

  @NotNull

  @GwtCompatibleEmail

  private String email;

  public Long getId() {

    return id;

  }

  public String getShortName() {

    return shortName;

  }

  public void setShortName(String shortName) {

    this.shortName = shortName;

  }

  public String getFullName() {

    return fullName;

  }

  public void setFullName(String fullName) {

    this.fullName = fullName;

  }

  public String getEmail() {

    return email;

  }

  public void setEmail(String email) {

    this.email = email;

  }

}

As you see, there are a lot of annotations on these model objects. These annotations correspond with the various roles these two classes play in the overall application. Clearly, these classes model a "To-do list" application where each user has 0 or more To-do items.

The annotations that apply to data synchronization are @Entity and @NamedQuery. These also apply to Errai JPA and server-side JPA, so there's actually no visible intrusion on the object model at all!

To use the data sync API, you talk to a class called ClientSyncManager. Here's an example from an Errai UI (client-side) class:

@Templated

public class TodoListSyncWidget extends Composite {

  @Inject private ClientSyncManager syncManager;

  @Inject private @DataField Label errorLabel;

  @Inject private @DataField Button syncButton;

  @EventHandler("syncButton")

  void sync(ClickEvent event) {

    Map<String,Object> params = new HashMap<String, Object>();

    params.put("user", user);

    syncManager.coldSync("allItemsForUser", TodoItem.class, params,

            new RemoteCallback<List<SyncResponse<TodoItem>>>() {

              @Override

              public void callback(List<SyncResponse<TodoItem>> response) {

                syncButton.setEnabled(true);

                System.out.println("Got data sync complete event!");

                refreshItems();

              }

            },

            new BusErrorCallback() {

              @Override

              public boolean error(Message message, Throwable throwable) {

                syncButton.setEnabled(true);

                errorLabel.setText("Sync failed: " + throwable);

                errorLabel.setVisible(true);

                return false;

              }

            });

    syncButton.setEnabled(false);

    System.out.println("Initiated cold sync");

  }

}

Walking through the above, here's all you need to do:

1. Inject an instance of ClientSyncManager.
2. Choose a JPA named query that you want to synchronize between client and server. In this case, we've chosen allItemsForUser.
3. Set the parameter values for the query you are using for the sync. In this case, the query takes a User object as a parameter.
4. Create a RemoteCallback<List<SyncResponse<YourModelType>>> which will be notified when the sync is complete. Note that your callback receives the list of data sync operations the client has just performed in reaction to the server's reply. You will normally ignore this (as the example above does) but you can also examine it if you'd like to trigger updates in your app based on certain objects being affected by the sync.
5. Optionally, create a BusErrorCallback to receive error messages associated with exchanging the sync request data with the server.
6. Invoke the ClientSyncManager.coldSync() method, passing it the items from steps 2-5.

The data sync sends changes from your client-side EntityManager (which stores its data in the browser's localStorage facility) to the server, which incorporates those changes and then responds with a list of changes the client needs to make to catch up with the server's current state. The client then makes the corresponding changes in your local EntityManager. All operations are confined to the results of the named query you specify, so you don't have to worry about syncing the entire server database to every client (unless your query is missing its WHERE clause!)

Before we move on to discuss communication with the server, why not try out the above demo? We've posted it to OpenShift so you can get a feel for how it works. Try signing in as the same user from multiple browsers (or maybe your phone or tablet) and get a feel for how the coldSync() call works. Remember: every time you press the Sync button, the client invokes ClientSyncManager.coldSync() as shown in the code snippet above.

Try the live demo on OpenShift now

Communicating with the Server

ClientSyncManager communicates with the server using an ErraiRPC Caller. In the current release, you are responsible for creating the server-side @Service class for Errai's Data Sync feature. This is for three reasons:

1. It gives you the opportunity to obtain the EntityManager from the correct server-side persistence context
2. It gives you the opportunity to screen and potentially reject sync requests (for example, requiring a logged-in user; disallowing sync requests that touch data that the current user is not allowed to see)
3. It allows you to choose an alternative transport mechanism (for example, Errai JAX-RS rather than ErraiBus)

We'd like to find a way that this will "just work" out-of-the-box, but still allow you to control the above-listed factors. Expect the following details to change before 3.0 goes final.

@ApplicationScoped @Service

public class DataSyncServiceImpl implements DataSyncService {

  @Inject private DataSyncEjb dataSyncEjb;

  @Override

  public <X> List<SyncResponse<X>> coldSync(

        SyncableDataSet<X> dataSet, List<SyncRequestOperation<X>> remoteResults) {

    // check prerequisites; throw security exceptions if they are not met...

    return dataSyncEjb.coldSync(dataSet, remoteResults);

  }

}

What's Next?

• server side per-object security callback, so you can vet each entity instance before it's sent to or accepted from the client
• dot notation in client-side JPQL queries, so your WHERE clause can refer to nested objects
• pruning results using the lazy fetch clause in the syncable named queries
• a client-side API for handling sync conflicts by performing a custom 3-way merge (the current release simply allows server state to win in case of a conflict)
• implementing application-managed transactions on the client side so you can choose to roll back when a conflict is encountered

And finally, incremental sync. The sync that's available now is a "cold" sync: it can be performed any time without any pre-existing contextual information on the server. You can perform cold syncs over and over to stay up to date. However, the cold sync needs to exchange a fair bit of data with the server in order to work. Our plan as we move forward is to implement an incremental sync which the cold sync process can hand off to. This will have two advantages over periodic cold syncs: it will save a lot of data transmission, and it will allow the server to push changes to the client instantly.

All the above is available both on 2.4.0.Beta1 and in the 3.0.0.20130604-M1 (read "3.0 milestone 1") releases. Update your errai.version property accordingly, and if your project depends on errai-javaee-all, you will have Errai Data Sync on your classpath! If you are not using errai-javaee-all, add the following dependency to your pom.xml:

    <dependency>

      <groupId>org.jboss.errai</groupId>

      <artifactId>errai-jpa-datasync</artifactId>

      <version>${errai.version}</version>

    </dependency>

Your feedback will help shape our path. Please join us on Freenode #errai, the errai-dev mailing list, or on our community forums.