I've written Resgate client in .NET!

General
#1

I’ve written Resgate client in .NET; Planned for my Unity (3D) based game.

Definitely not completed, but I will get back to it if I get some time :wink:

Feel free to fork or submit pull requests.

I will push to nugets later on

#2

That is awesome!

This is something that @novagen should be interested very in, as they were also using Unity together with Resgate. I will make sure he is properly notified.

Great work! And if you need someone to bounce ideas with, I’m more than happy to assist :grin:.

/Samuel

#3

Actually there is something still ahead of myself - authentication (and my subscription model that survives re-connection via resubscribing and ignoring the same resources returned - if they are the same, of course :wink: ).

I wonder if there is a way to re-authenticate after connection loss safely with minimal user interaction (for example token used for auth might be invalid for re-submitting). I believe the safest bet would be to provide callback (“Please, re-authenticate!”) to the API user, but that way I am blocking my threads until authentication is done (or cancelled). For example you won’t get subscriptions updates for guest-accessable resources till you either accept or cancel that auth request. There might be a problem with calls also, as I do not have the way to check whenever authentication is required for call or not.

Or perhaps I’m overthinking there, and there is a safer road…?

#4

Yes and no. But yes. :slight_smile:
Resgate will drop all client state on disconnect, so it is up to the client to reconnect and reauthenticate.

In the Javascript client, ResClient, this is done through the setOnConnect callback.

function relogin() {
	let reloginKey = localStorage.getItem('reloginKey');
	if (!reloginKey) return;

	return client.authenticate('session', 'relogin', { reloginKey }).then(result => {
		localStorage.setItem("reloginKey", result.reloginKey);
	}).catch(err => { /* ... */ });
}

client.setOnConnect(relogin);

The idea is that, on initial login, you get back some sort of token that can be used for reauthentication, to prevent the user from having to enter credentials again.

This reloginKey is just some kind of token that can be used as credentials instead of username/password. Maybe a JWT bearer token issued by some oauth2 provider? Maybe just some string or short-lived JWT token that your login-service returned when the user logged in.

So, on disconnect, the Javascript ResClient will do this:

  1. Get involuntarily disconnected. All resources are kept in cache, but gets marked as stale.
  2. Periodically try to reconnect to any Resgate
  3. On successful reconnect:
    1. Send the version request
    2. Call the setOnConnect callback (if one is set), and wait until that is completed (promise is resolved). This is where the user is reauthenticated with some stored token.
    3. Subscribe to all the stale resources that was subscribed prior to the disconnect
    4. Compare the response from 3) with the cached resources, and locally generate/emit “fake” events that describe the diff.
  4. All is resynchronized! The user never even had to notice it ever happening.

If you want to see this in action, you could try the resgate-test-app which has an auth part that does this. Try run it (simple node.js app):

  • Open the app in Chrome
  • go to the Auth tab
  • In Chrome Developer Tools, check the Resgate WebSocket messages:
  • login with the credentials showing on that page.
  • maybe go to the Ticker tab
  • shut down Resgate to force a disconnect
  • restart Resgate
  • See how the Ticker is resynchronized, and you are reauthenticated.

/Samuel

#5

Thanks Samuel, it lighten up things! Seems it covers my idea of callback on re-connection :slight_smile:

Just two more questions:
1.) If API user wants to make a “Get” call to an already subscribed resource is it enough to return client-side cached value?

2.) I saw in docs that “Call” requests may return rid and I should consider them directly subscribed. Can you elaborate a bit more on that? What are the use-cases?

#6

Answer to 1

Yes, no need to fetch it again. In fact, since Resgate knows it is subscribed by the client, it will not even include it in the returned resource set, if you were to send a request:

Any request or event resulting in new subscriptions will contain a set of resources that contains any subscribed resource previously not subscribed by the client.

But, a little “warning” if that resource is indirectly subscribed, a.k.a. was referenced by a subscribed resource, then you might want to do differently. Let’s assume you have these models:

  • example.a : { "refToB": { "rid": "example.b" }}
  • example.b : { "foo": "bar"}

If you first subscribe to example.a, Resgate will include example.b in the returned resource set, and it will be seen as indirectly subscribed.
If the API user wants to get example.b, you should should be aware that, if example.a is unsubscribed, the indirect subscription to example.b will also be gone. So, if you want to keep example.b’s subscription, you need to send a separate subscribe request for example.b.

ResClient does not send any unnecessary subscribes. If example.a would be unsubscribed while some UI component is still listening on example.b, ResClient will mark example.b as stale (as the indirect subscription is lost), send a subscribe for example.b, and when it gets the response, it will compare the result with the cached value. If there are difference, it will generate events in the same way as when re-synchronizing on a reconnect.

Answer to 2

I will quote from another answer I just made :slight_smile:

The resource response is to make method calls that return a resource. This is commonly used when making calls to create resources, as you often want to get that newly created resource back:

client.call('inventory.items', 'create', { name: "Rubber duck", quantity: 20 })
    .then(item => console.log(item.name)); // Logs Rubber duck

The client doesn’t know before-hand the resource ID of the newly created resource, but when the service responds with the resource ID, Resgate will fetch that resource and return it to the client in the response.

So you can also use it for letting clients subscribe to resources they don’t know the resource ID for. Maybe, after logging in, you want to get your own user profile, without knowing your own user ID. Eg:

client.call('directory.users', 'getCurrentlyLoggedInUser')
    .then(user => console.log(user.name)); // Samuel

Basically, if you get an rid instead of a payload in the call (or auth) response, you can treat the response as if it was a response to a subscribe request for that resource ID in the rid field.

And if you get more questions, just bring them on! :grin:
Always happy to help out in client-writing endeavors.

/Samuel

#7

Ah, I see!

I actually think I got a little overboard in case of first - I compose entire object on-the-fly (supporting cyclic references) and return it to the user. So in the example the returned type would be:

class ExampleA
{
   public ExampleB refToB;
}
class ExampleB
{
    public string foo;
}

The object ExampleA is populated as initial. Then if B change, the library will know it was indirect subscription and lookups all direct subscriptions using it, triggering changed event on them with entire new object ExampleA passed as argument. I must think for better solution for large objects through… :slight_smile:

#8

Sounds like an interesting approach, and I think a good one.

So, if foo changes, you will create both a new ExampleA and ExampleB?

And if you have a ExampleC that also references ExampleB:

class ExampleC
{
   public ExampleB refToB;
}

You would also create a new ExampleC?

In my pipeline of things to do, I have the task of writing a Resgate client for Go.
While thinking about how to do it, I also ended up thinking that I must try to work with “immutable” data (well, can’t really do immutables in Go, but you can tell developers not to mutate them!). So that, when a change happens, I would create new entities. Just like you are describing.
My option would otherwise be to go down the path of single mutating entities protected by mutexes and locks… which I believe would be hard to work with.

And yes, for large models or collections, this might be an issue. But, large objects are anyway discouraged, and can often be broken into parts or made partial (like pagination).

And sorry about me letting the RES client protocol allow cyclic references. Sometimes I wonder if that was a poor design choice as it added quite a bit to the complexity when writing the client. But then again, it can be a fun task trying to implement it :grin:

#9

Yes, exactly - and the new ExampleC will be created as well.

The algorithm follows:

First, let us use specific data container, called BijectiveDictionary - it is many-to-many container, indexable both ways. It wastes some space, but has very fast lookup of single-to-many objects both ways (actually the same lookup speed as Dictionary in C#, which Microsoft describes as “approaches O(1)” - believe or not :slight_smile: )

So in the left side we have all currently subscribed models (both direct and indirect) and right side is all items that are used by that object.

So if we subscribe object ExampleA it will:

  1. Create queue for objects
  2. Add all direct subobjects of ExampleA into queue (in this case only ExampleB)
  3. While queue not empty:
    3A. Pop element from queue
    3B. If element not registered as subobject of ExampleA
    -> Add pair ExampleA, element item to subobject bi-dictionary
    -> Add all direct subobjects of element item (which are not already registered as subobjects) to queue

So in this case we will have bi-dictionary of:
ExampleA -> ExampleB
ExampleC -> ExampleB

And querying ExampleB reverse way will yield the result of both ExampleA and ExampleC.

Now, if we know that ExampleB was modified via event we do:

  1. Rebuild ExampleB (see below) and trigger changed event for ExampleB (if it is indirect, there will be no listeners for the events)
  2. For each object that is paired from the left to ExampleB on the right in bi-dictionary (all objects that uses ExampleB, even via any amount of layers of indirection)
    2A. Rebuild that object (see below) and trigger changed event for that object

Rebuild algorithm:

  1. Keep a Dictionary of rid -> created object
  2. Create current object
  3. Add it to Dictionary of rid -> created object
  4. If current object contains rid:
    3A. check if objects is already in Dictionary, if so - populate it from it
    3B. if not, recursively (or via queue) go to 2.)

It is immutable, but you are right - the large object will be rare, so I think I’ll keep it that way. Oh, and cyclic references actually allows much more freedom for server writers, I don’t think it is a bad design decision after all :smiley:

#10

Wow.

I really like this! :100:

It seems to simplify what I had quite a bit of struggle with when doing the Javascript client.
The Javascript client only keeps a single direction reference from a resource to all the referenced “children”, together with counters on how many times a resource is indirect/direct referenced.

Whenever something changes (due to subscribes/unsubscribes or change/add/remove events), these references are recursively traversed, the counters changed, and any resource reaching 0 counts are evicted (or just marked as stale, if they are still needed, in which case they will be resubscribed).

But that recursive traverse together with possible cyclic reference, was a bit tricky to implement. Easy to make logical mistakes.

I will look more into your approach!

And, glad you think it was a good choice allowing cyclic refs :grin: . It gives more freedom, after all.

#11

Thanks for kind words :slight_smile:

Just for the sake of completeness:

  • BijectiveDictionary is implemented as two ordinary dictionaries Left -> Set<Right> and Right -> Set<Left> packed into class, so they don’t go out of sync
  • This also mean, the dictionary does not allows duplicates (each pair can be stored only once). Of course pairs Obj17, Obj32 and Obj32, Obj17 are not duplicates (order is important)
  • Unregistering directly-referenced object removes each instance from the left and from the right
  • Indirectly referenced object should die automatically with this approach
  • This is NOT YET BATTLE-PROVEN. I actually designed this being a bit ill (not corona, but still) and still did not check all the corner cases :slight_smile:

Oh, and I need to re-write auth part. After that - and after testing some corner cases for registration - the library should be ready for Release Candidate version and be included in NPM :slight_smile:

#12

From what I’ve seen, it looks both well structured and really promising.

I haven’t really gone into its logic to see if it works in battle, only been trying out the examples, and checking out the API.
But when time is given (at the moment I am trying to add some community requested features), I wish to look much closer at it, and see if I can help in any way.

Speaking about examples, I posted issue #1 on the GitHub repository for Example1 :grin:

If it is all right with you, I’d be glad to add this .NET client to Resgate.io - Resources.

#13

Fixed, thanks! :slight_smile:

I need to finish up API for auth, I don’t like the form it has now. Also some more general error checking (like custom callback for Get methods, alongside with existing Subscriptions) would be nice. After I’m done with that I’ll be more than happy to have this added into .NET client to resources.

1 Like
#14

… real life can be really tricky these times :smiley:

After 1.5 years I’ve circled around different tech-stacks and projects and I’m back to the place where I need again a Resgate client in C#. And guess what? The development of my utility will resume shortly! Expect bringing it up to date with current Resgate protocol version within days, fixing error checking and some touches for auth api as well.

After I’m done with that I will add it to Nugets and be very grateful for adding to the official resources list. Just give me a week or two to process everything :slight_smile:

Merry Christmas!