Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
1.2k views
in Technique[技术] by (71.8m points)

select - Get last unique JSON inputs in bash

I'm looking to pull the latest (by timestamp) source and message from some JSON data. I tried to get jq to pull only the latest unique message based on the source but can't quite figure it out.

Example source data is:

{"envelope":{"source":"+13101234567","sourceDevice":1,"relay":null,"timestamp":1610256979995,"dataMessage":{"timestamp":1610256979995,"message":"Yes","expiresInSeconds":0,"reaction":null,"quote":null,"mentions":[],"attachments":[],"groupInfo":null},"syncMessage":null,"callMessage":null,"receiptMessage":null}}
{"envelope":{"source":"+13102345678","sourceDevice":1,"relay":null,"timestamp":1610256985623,"dataMessage":{"timestamp":1610256985623,"message":"1","expiresInSeconds":0,"reaction":null,"quote":null,"mentions":[],"attachments":[],"groupInfo":null},"syncMessage":null,"callMessage":null,"receiptMessage":null}}
{"envelope":{"source":"+13102345678","sourceDevice":1,"relay":null,"timestamp":1610256987736,"dataMessage":{"timestamp":1610256987736,"message":"3","expiresInSeconds":0,"reaction":null,"quote":null,"mentions":[],"attachments":[],"groupInfo":null},"syncMessage":null,"callMessage":null,"receiptMessage":null}}
{"envelope":{"source":"+13101234567","sourceDevice":1,"relay":null,"timestamp":1610256990731,"dataMessage":{"timestamp":1610256990731,"message":"4","expiresInSeconds":0,"reaction":null,"quote":null,"mentions":[],"attachments":[],"groupInfo":null},"syncMessage":null,"callMessage":null,"receiptMessage":null}}
{"envelope":{"source":"+13105678901","sourceDevice":1,"relay":null,"timestamp":1610256990731,"dataMessage":{"timestamp":1610256990731,"message":"4","expiresInSeconds":0,"reaction":null,"quote":null,"mentions":[],"attachments":[],"groupInfo":null},"syncMessage":null,"callMessage":null,"receiptMessage":null}}

For this example, the result should be:

[{"source":"+13102345678","message":"3"},{"source":"+13101234567","message":"4"},{"source":"+13105678901","message":"4"}]

Using .envelope.source,.envelope.dataMessage.message gets the information without the headers but piping it into unique[] says "Cannot iterate over string" on jqplay.

question from:https://stackoverflow.com/questions/65650703/get-last-unique-json-inputs-in-bash

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

As I understand the question, a reasonable solution would be as follows, assuming the -n command-line option is used:

[inputs.envelope
 | {source, 
    message: .dataMessage.message,
    timestamp: .dataMessage.timestamp}]
| group_by(.source)
| [ .[] | max_by(.timestamp) | {source, message} ]

This at least produces the expected results, and pays attention to the "timestamps".


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...