Models and collections are scary

General
#21

Two possible reasons. It is probably nr 2, if you use go-res:

  1. It must be valid JSON. JSON only allows string keys. go-res produces valid JSON though.
  2. You cannot have an object as value on a model ({"from": "someone", "to": "someone else"}). Sorry, but it is the nested thing again :sweat_smile:.

If it is 2, and you want the value to be structured JSON, you can “solve” it by JSON encoding the value and add it as a string:

{
    "1": "{\"from\":\"someone\":,\"to\":\"someone else\"}",
    "2": "{\"from\":\"another\":,\"to\":\"yet another\"}"
}

And use JSON.parse(model[1]) to get the value on the client.

POST EDIT:

Hmm, if more people end up needing something like this, maybe I should consider a new “data” value type, which allows arbitrary data. Eg.:

{
    "1": { "data": {"from": "someone", "to": "someone else"}}
}
#22

hmm, i see.

I’m guess I’m still trying to wrap my head around models, right now I’m building a simple conditional message system, e.g if condition A, send message to someone, which seems to be turning into a full blown message system because tomorrow i plan to add responses.

I don’t actually need to use integers for the keys i just needed something unique for the bolt key value store and they seemed logical but when i added them everything just stopped working :frowning:

also on a somewhat related note i noticed that the resgate JavaScript sdk .on(‘change’) listener tends to send a lot of access requests randomly with no tokens, it doesn’t seem to be hurting anything but it is filling up my logs… once i get this model pattern sorted out I’ll dig into it to see what i can find.

#23

Without seeing any logs, I can only guess.

My guess is that you fetch some list of resource references to a bunch of messages. Granting access to the list, the RES protocol indirectly gives access to all resources that is referred to by that list, a.k.a. all the messages.

And then you use .on('change') on each message, but you don’t use .on on the list itself.

What happens then is that, because ResClient has no event listeners on the list itself, it will eventually unsubscribe to the list. This will cause the all indirectly subscribed messages also to be unsubscribed. ResClient will therefore subscribe to each message that has some .on-listener separately. Which in turn will make Resgate send access requests for those message (but no get request as the data is already in the cache).

Then ResClient unsubscribes to the list.

If I am right, all you have to do is to add an event listener to the list itself:

client.get("messages.list", list => {
    list.on(); // An empty listener is fine. It tells ResClient not to unsubscribe to the list.
    /* ... more stuff ... */
});
#24

If by conditional you mean being able to send messages to different people without others being able to see the messages, then you should think of a resource as “channel”. Each user has its own “channel” resource:

s.Handle("user.$userId.inbox", /* ... */)

In the Access handler, you make sure only that user can access this resource.

Then, to send a message to that user, you can make a call handler like this:

service.Handle("user.$userId", res.Call("send", func(r res.CallRequest) {
	var p struct {
		ToUserID string `json:"toUserId"`
		Message  string `json:"message"`
	}
	r.ParseParams(&p)

	/* ... validate toUserId ... */

	// Get the inbox resource context of the user we are sending to
	r.Service().With("msgservice.user."+p.ToUserID+".inbox", func(re res.Resource) {
		// If your "inbox" is a model
		re.ChangeEvent(map[string]interface{}{
			// I use timestamp as key. Bad if two messages are sent at the same nanosecond ;)
			strconv.FormatInt(time.Now().UnixNano(), 10): p.Message,
		})

		// If your "inbox" is a collection. I just add a new message on top.
		re.AddEvent(p.Message, 0)

		// Or if you use custom events for sending. These will not be cached by Resgate though.
		re.Event("incomingMessage", struct {
			From    string `json:"from"`
			Message string `json:"message"`
		}{r.PathParam("userId"), p.Message})
	})
}))

Not sure if that helped.
But that is one way to do it :slight_smile:

1 Like
#25

this got me thinking it wasn’t the key causing the problem but the value (well that and the fact you said the value had to be a string) so I tried marshalling the payload and viola it worked

err := nh.db.Update(func(tx *bolt.Tx) error {
		b := tx.Bucket([]byte(address))
		id, _ := b.NextSequence()
		data, err := json.Marshal(param) //<-- this is what did the trick
		if err != nil {
			return err
		}
		err = b.Put([]byte(strconv.Itoa(int(id))), []byte(r.RawParams()))
		event[strconv.Itoa(int(id))] = string(data)
		return err
	})

which makes sense now, the resgate error wasn’t lying to me :stuck_out_tongue:

2020/06/10 15:37:21.267411 [ERR] Error processing event system.account.notification.GBIB5ILR5VGHUG2LZVHXRSBGJ7GSTK4UAEYAGENOK6FK43UN6ZP2UHML.change: Internal error: invalid value

Here’s the whole service for anyone interested

#26

I ran into the problem with returning nested structures from the QueryModel again…
any chance there has been some movement on allowing nested data structures?

#27

Maybe.

The data value that was introduced in Resgate v1.6.0 (RES protocol v1.2.1) has covered the needs for nested structures (that are either immutable or semi-static) in our own projects.

Example:

To describe the following nested structure:

{
    "arr": [1, 2, 3],
    "obj": { "id": 42, "payload": { "msg": "hello" } }
}

We can use a model with data values. A response to a get request would look like this:

get.example.model

{
    "result": {
        "model": {
            "arr": { "data": [1, 2, 3] },
            "obj": { "data": { "id": 42, "payload": { "msg": "hello" } } }
        }
    }
}

The drawback is that, in order to change the "hello" message, you must send the entire "obj" data value in the change event:

event.example.model.change

{
    "values": {
        "obj": { "data": { "id": 42, "payload": { "msg": "hejsan" } } }
    }
}

Would that help?

Best regards,
Samuel

1 Like
#28

Yes that seems like it would solve the issue we’ve been dealing with.