From 3086a205bef0f1e5966cc58984c7e86f375523c9 Mon Sep 17 00:00:00 2001 From: dskvr Date: Mon, 15 Jul 2024 14:57:34 +0200 Subject: [PATCH 1/4] NIP-119 --- 119.md | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 119.md diff --git a/119.md b/119.md new file mode 100644 index 00000000..1481533e --- /dev/null +++ b/119.md @@ -0,0 +1,38 @@ +# NIP-119: AND Operator in Filters + +Enable `AND` within a single tag filter by using an `&` modifier in filters for indexable tags. + +``` +filters: { + "kinds": [1], + "&t": ["meme", "cat"], + "#t": ["black", "white"] +} +// returns kind `1` events with `t` tags that have both "meme" and "cat" that have the tag "black" or "white" +``` + +## Rationale +- Reduce bandwidth for all, with an emphasis on mobile users: `meme AND cat` objectively consumes less bandwidth than `meme OR cat ` +- Reduce clock-time for relays, indexing with `AND` is fast for all common index formats, and faster compared to `OR` for some index formats. (See section below) +- Reduce client-side caching requirements +- Reduce centralization vectors by reducing or even eliminating the need for centralized REST, GraphQL APIs or specialized relay "feed" endpoints. +- Give relays the option to be more useful at the protocol level while improving efficiency for all parties. + +## Rules + +- `AND` **MUST** take precedence over `OR` +- Tags used in `AND` **SHOULD NOT** be used in standard `OR` tags [`#`] +- Any tag used in `AND` **SHOULD** be ignored in `OR` + +## Considerations + +- New field for `NIP-11.limitations`: `max_tags_per_and` and `max_tags_and` +- Benchmarking should be conducted to validate that bandwidth and protocol usability as benefits supercede implementation and clock-time cost. + +## Index Efficiency +| Index Type | AND Operation Efficiency | OR Operation Efficiency | Notes | +|----------------|--------------------------|-------------------------|-------| +| B-Tree | High | Moderate | B-Tree indexes are very efficient for AND operations, especially with compound indexes. For OR operations, they are less efficient than for AND, as the database engine might need to traverse multiple paths. | +| Bitmap | High | High | Bitmap indexes excel in both AND and OR operations, particularly for columns with low cardinality. They utilize fast bitwise operations, making them ideal for read-heavy environments. | +| Hash | Not Applicable | Not Applicable | Hash indexes are designed for equality checks and do not directly support range-based queries or optimize for AND/OR operations efficiently. | +| Full-Text | High | High | Optimized for text search, full-text indexes efficiently handle both AND and OR conditions, making them suitable for complex text queries. | \ No newline at end of file From b0491854f89dbd841c122ac139673c9a9b93dc64 Mon Sep 17 00:00:00 2001 From: dskvr Date: Thu, 29 Aug 2024 16:55:30 +0200 Subject: [PATCH 2/4] remove 'considerations' and 'index efficiency' --- 119.md | 15 +-------------- 1 file changed, 1 insertion(+), 14 deletions(-) diff --git a/119.md b/119.md index 1481533e..d5fe5983 100644 --- a/119.md +++ b/119.md @@ -22,17 +22,4 @@ filters: { - `AND` **MUST** take precedence over `OR` - Tags used in `AND` **SHOULD NOT** be used in standard `OR` tags [`#`] -- Any tag used in `AND` **SHOULD** be ignored in `OR` - -## Considerations - -- New field for `NIP-11.limitations`: `max_tags_per_and` and `max_tags_and` -- Benchmarking should be conducted to validate that bandwidth and protocol usability as benefits supercede implementation and clock-time cost. - -## Index Efficiency -| Index Type | AND Operation Efficiency | OR Operation Efficiency | Notes | -|----------------|--------------------------|-------------------------|-------| -| B-Tree | High | Moderate | B-Tree indexes are very efficient for AND operations, especially with compound indexes. For OR operations, they are less efficient than for AND, as the database engine might need to traverse multiple paths. | -| Bitmap | High | High | Bitmap indexes excel in both AND and OR operations, particularly for columns with low cardinality. They utilize fast bitwise operations, making them ideal for read-heavy environments. | -| Hash | Not Applicable | Not Applicable | Hash indexes are designed for equality checks and do not directly support range-based queries or optimize for AND/OR operations efficiently. | -| Full-Text | High | High | Optimized for text search, full-text indexes efficiently handle both AND and OR conditions, making them suitable for complex text queries. | \ No newline at end of file +- Any tag used in `AND` **SHOULD** be ignored in `OR` \ No newline at end of file From c7da9d7409f25cdb34be1ea7d7aded259cb65e7f Mon Sep 17 00:00:00 2001 From: Sandwich <299465+dskvr@users.noreply.github.com> Date: Mon, 23 Sep 2024 23:43:48 +0200 Subject: [PATCH 3/4] remove rationale --- 119.md | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/119.md b/119.md index d5fe5983..3ab120a2 100644 --- a/119.md +++ b/119.md @@ -11,15 +11,8 @@ filters: { // returns kind `1` events with `t` tags that have both "meme" and "cat" that have the tag "black" or "white" ``` -## Rationale -- Reduce bandwidth for all, with an emphasis on mobile users: `meme AND cat` objectively consumes less bandwidth than `meme OR cat ` -- Reduce clock-time for relays, indexing with `AND` is fast for all common index formats, and faster compared to `OR` for some index formats. (See section below) -- Reduce client-side caching requirements -- Reduce centralization vectors by reducing or even eliminating the need for centralized REST, GraphQL APIs or specialized relay "feed" endpoints. -- Give relays the option to be more useful at the protocol level while improving efficiency for all parties. - ## Rules - `AND` **MUST** take precedence over `OR` - Tags used in `AND` **SHOULD NOT** be used in standard `OR` tags [`#`] -- Any tag used in `AND` **SHOULD** be ignored in `OR` \ No newline at end of file +- Any tag used in `AND` **SHOULD** be ignored in `OR` From b792e36eebc4df0464d5290c3dda20f38a3d7242 Mon Sep 17 00:00:00 2001 From: Sandwich <299465+dskvr@users.noreply.github.com> Date: Fri, 18 Oct 2024 10:47:57 +0200 Subject: [PATCH 4/4] Update 119.md --- 119.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/119.md b/119.md index 3ab120a2..df54085d 100644 --- a/119.md +++ b/119.md @@ -14,5 +14,5 @@ filters: { ## Rules - `AND` **MUST** take precedence over `OR` -- Tags used in `AND` **SHOULD NOT** be used in standard `OR` tags [`#`] -- Any tag used in `AND` **SHOULD** be ignored in `OR` +- Tag values used in `AND` **SHOULD NOT** be used in standard `OR` tags [`#`] +- Any tag value used in `AND` **SHOULD** be ignored in `OR`