66 lines
18 KiB
HTML
66 lines
18 KiB
HTML
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><meta name="generator" content="rustdoc"><meta name="description" content="This trait allows the user to customize the heuristic used to determine the relative frequency of a given byte in the dataset being searched."><title>HeuristicFrequencyRank in memchr::arch::all::packedpair - Rust</title><script>if(window.location.protocol!=="file:")document.head.insertAdjacentHTML("beforeend","SourceSerif4-Regular-6b053e98.ttf.woff2,FiraSans-Italic-81dc35de.woff2,FiraSans-Regular-0fe48ade.woff2,FiraSans-MediumItalic-ccf7e434.woff2,FiraSans-Medium-e1aa3f0a.woff2,SourceCodePro-Regular-8badfe75.ttf.woff2,SourceCodePro-Semibold-aa29a496.ttf.woff2".split(",").map(f=>`<link rel="preload" as="font" type="font/woff2"href="../../../../static.files/${f}">`).join(""))</script><link rel="stylesheet" href="../../../../static.files/normalize-9960930a.css"><link rel="stylesheet" href="../../../../static.files/rustdoc-ca0dd0c4.css"><meta name="rustdoc-vars" data-root-path="../../../../" data-static-root-path="../../../../static.files/" data-current-crate="memchr" data-themes="" data-resource-suffix="" data-rustdoc-version="1.93.1 (01f6ddf75 2026-02-11) (Arch Linux rust 1:1.93.1-1)" data-channel="1.93.1" data-search-js="search-9e2438ea.js" data-stringdex-js="stringdex-a3946164.js" data-settings-js="settings-c38705f0.js" ><script src="../../../../static.files/storage-e2aeef58.js"></script><script defer src="sidebar-items.js"></script><script defer src="../../../../static.files/main-a410ff4d.js"></script><noscript><link rel="stylesheet" href="../../../../static.files/noscript-263c88ec.css"></noscript><link rel="alternate icon" type="image/png" href="../../../../static.files/favicon-32x32-eab170b8.png"><link rel="icon" type="image/svg+xml" href="../../../../static.files/favicon-044be391.svg"></head><body class="rustdoc trait"><!--[if lte IE 11]><div class="warning">This old browser is unsupported and will most likely display funky things.</div><![endif]--><rustdoc-topbar><h2><a href="#">HeuristicFrequencyRank</a></h2></rustdoc-topbar><nav class="sidebar"><div class="sidebar-crate"><h2><a href="../../../../memchr/index.html">memchr</a><span class="version">2.8.0</span></h2></div><div class="sidebar-elems"><section id="rustdoc-toc"><h2 class="location"><a href="#">Heuristic<wbr>Frequency<wbr>Rank</a></h2><h3><a href="#">Sections</a></h3><ul class="block top-toc"><li><a href="#example" title="Example">Example</a></li></ul><h3><a href="#required-methods">Required Methods</a></h3><ul class="block"><li><a href="#tymethod.rank" title="rank">rank</a></li></ul><h3><a href="#foreign-impls">Implementations on Foreign Types</a></h3><ul class="block"><li><a href="#impl-HeuristicFrequencyRank-for-%26R" title="&'a R">&'a R</a></li></ul><h3><a href="#implementors">Implementors</a></h3></section><div id="rustdoc-modnav"><h2><a href="index.html">In memchr::<wbr>arch::<wbr>all::<wbr>packedpair</a></h2></div></div></nav><div class="sidebar-resizer" title="Drag to resize sidebar"></div><main><div class="width-limiter"><section id="main-content" class="content"><div class="main-heading"><div class="rustdoc-breadcrumbs"><a href="../../../index.html">memchr</a>::<wbr><a href="../../index.html">arch</a>::<wbr><a href="../index.html">all</a>::<wbr><a href="index.html">packedpair</a></div><h1>Trait <span class="trait">Heuristic<wbr>Frequency<wbr>Rank</span> <button id="copy-path" title="Copy item path to clipboard">Copy item path</button></h1><rustdoc-toolbar></rustdoc-toolbar><span class="sub-heading"><a class="src" href="../../../../src/memchr/arch/all/packedpair/mod.rs.html#309-318">Source</a> </span></div><pre class="rust item-decl"><code>pub trait HeuristicFrequencyRank {
|
|
// Required method
|
|
fn <a href="#tymethod.rank" class="fn">rank</a>(&self, byte: <a class="primitive" href="https://doc.rust-lang.org/1.93.1/std/primitive.u8.html">u8</a>) -> <a class="primitive" href="https://doc.rust-lang.org/1.93.1/std/primitive.u8.html">u8</a>;
|
|
}</code></pre><details class="toggle top-doc" open><summary class="hideme"><span>Expand description</span></summary><div class="docblock"><p>This trait allows the user to customize the heuristic used to determine the
|
|
relative frequency of a given byte in the dataset being searched.</p>
|
|
<p>The use of this trait can have a dramatic impact on performance depending
|
|
on the type of data being searched. The details of why are explained in the
|
|
docs of <a href="../../../memmem/enum.Prefilter.html" title="enum memchr::memmem::Prefilter"><code>crate::memmem::Prefilter</code></a>. To summarize, the core algorithm uses
|
|
a prefilter to quickly identify candidate matches that are later verified
|
|
more slowly. This prefilter is implemented in terms of trying to find
|
|
<code>rare</code> bytes at specific offsets that will occur less frequently in the
|
|
dataset. While the concept of a <code>rare</code> byte is similar for most datasets,
|
|
there are some specific datasets (like binary executables) that have
|
|
dramatically different byte distributions. For these datasets customizing
|
|
the byte frequency heuristic can have a massive impact on performance, and
|
|
might even need to be done at runtime.</p>
|
|
<p>The default implementation of <code>HeuristicFrequencyRank</code> reads from the
|
|
static frequency table defined in <code>src/memmem/byte_frequencies.rs</code>. This
|
|
is optimal for most inputs, so if you are unsure of the impact of using a
|
|
custom <code>HeuristicFrequencyRank</code> you should probably just use the default.</p>
|
|
<h2 id="example"><a class="doc-anchor" href="#example">§</a>Example</h2>
|
|
<div class="example-wrap"><pre class="rust rust-example-rendered"><code><span class="kw">use </span>memchr::{
|
|
arch::all::packedpair::HeuristicFrequencyRank,
|
|
memmem::FinderBuilder,
|
|
};
|
|
|
|
<span class="doccomment">/// A byte-frequency table that is good for scanning binary executables.
|
|
</span><span class="kw">struct </span>Binary;
|
|
|
|
<span class="kw">impl </span>HeuristicFrequencyRank <span class="kw">for </span>Binary {
|
|
<span class="kw">fn </span>rank(<span class="kw-2">&</span><span class="self">self</span>, byte: u8) -> u8 {
|
|
<span class="kw">const </span>TABLE: [u8; <span class="number">256</span>] = [
|
|
<span class="number">255</span>, <span class="number">128</span>, <span class="number">61</span>, <span class="number">43</span>, <span class="number">50</span>, <span class="number">41</span>, <span class="number">27</span>, <span class="number">28</span>, <span class="number">57</span>, <span class="number">15</span>, <span class="number">21</span>, <span class="number">13</span>, <span class="number">24</span>, <span class="number">17</span>, <span class="number">17</span>,
|
|
<span class="number">89</span>, <span class="number">58</span>, <span class="number">16</span>, <span class="number">11</span>, <span class="number">7</span>, <span class="number">14</span>, <span class="number">23</span>, <span class="number">7</span>, <span class="number">6</span>, <span class="number">24</span>, <span class="number">9</span>, <span class="number">6</span>, <span class="number">5</span>, <span class="number">9</span>, <span class="number">4</span>, <span class="number">7</span>, <span class="number">16</span>,
|
|
<span class="number">68</span>, <span class="number">11</span>, <span class="number">9</span>, <span class="number">6</span>, <span class="number">88</span>, <span class="number">7</span>, <span class="number">4</span>, <span class="number">4</span>, <span class="number">23</span>, <span class="number">9</span>, <span class="number">4</span>, <span class="number">8</span>, <span class="number">8</span>, <span class="number">5</span>, <span class="number">10</span>, <span class="number">4</span>, <span class="number">30</span>, <span class="number">11</span>,
|
|
<span class="number">9</span>, <span class="number">24</span>, <span class="number">11</span>, <span class="number">5</span>, <span class="number">5</span>, <span class="number">5</span>, <span class="number">19</span>, <span class="number">11</span>, <span class="number">6</span>, <span class="number">17</span>, <span class="number">9</span>, <span class="number">9</span>, <span class="number">6</span>, <span class="number">8</span>,
|
|
<span class="number">48</span>, <span class="number">58</span>, <span class="number">11</span>, <span class="number">14</span>, <span class="number">53</span>, <span class="number">40</span>, <span class="number">9</span>, <span class="number">9</span>, <span class="number">254</span>, <span class="number">35</span>, <span class="number">3</span>, <span class="number">6</span>, <span class="number">52</span>, <span class="number">23</span>, <span class="number">6</span>, <span class="number">6</span>, <span class="number">27</span>,
|
|
<span class="number">4</span>, <span class="number">7</span>, <span class="number">11</span>, <span class="number">14</span>, <span class="number">13</span>, <span class="number">10</span>, <span class="number">11</span>, <span class="number">11</span>, <span class="number">5</span>, <span class="number">2</span>, <span class="number">10</span>, <span class="number">16</span>, <span class="number">12</span>, <span class="number">6</span>, <span class="number">19</span>,
|
|
<span class="number">19</span>, <span class="number">20</span>, <span class="number">5</span>, <span class="number">14</span>, <span class="number">16</span>, <span class="number">31</span>, <span class="number">19</span>, <span class="number">7</span>, <span class="number">14</span>, <span class="number">20</span>, <span class="number">4</span>, <span class="number">4</span>, <span class="number">19</span>, <span class="number">8</span>, <span class="number">18</span>, <span class="number">20</span>, <span class="number">24</span>,
|
|
<span class="number">1</span>, <span class="number">25</span>, <span class="number">19</span>, <span class="number">58</span>, <span class="number">29</span>, <span class="number">10</span>, <span class="number">5</span>, <span class="number">15</span>, <span class="number">20</span>, <span class="number">2</span>, <span class="number">2</span>, <span class="number">9</span>, <span class="number">4</span>, <span class="number">3</span>, <span class="number">5</span>,
|
|
<span class="number">51</span>, <span class="number">11</span>, <span class="number">4</span>, <span class="number">53</span>, <span class="number">23</span>, <span class="number">39</span>, <span class="number">6</span>, <span class="number">4</span>, <span class="number">13</span>, <span class="number">81</span>, <span class="number">4</span>, <span class="number">186</span>, <span class="number">5</span>, <span class="number">67</span>, <span class="number">3</span>, <span class="number">2</span>, <span class="number">15</span>,
|
|
<span class="number">0</span>, <span class="number">0</span>, <span class="number">1</span>, <span class="number">3</span>, <span class="number">2</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">5</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">2</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>,
|
|
<span class="number">12</span>, <span class="number">2</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">3</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">6</span>, <span class="number">1</span>, <span class="number">2</span>, <span class="number">1</span>, <span class="number">3</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">2</span>, <span class="number">9</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">0</span>,
|
|
<span class="number">2</span>, <span class="number">2</span>, <span class="number">4</span>, <span class="number">4</span>, <span class="number">11</span>, <span class="number">6</span>, <span class="number">7</span>, <span class="number">3</span>, <span class="number">6</span>, <span class="number">9</span>, <span class="number">4</span>, <span class="number">5</span>,
|
|
<span class="number">46</span>, <span class="number">18</span>, <span class="number">8</span>, <span class="number">18</span>, <span class="number">17</span>, <span class="number">3</span>, <span class="number">8</span>, <span class="number">20</span>, <span class="number">16</span>, <span class="number">10</span>, <span class="number">3</span>, <span class="number">7</span>, <span class="number">175</span>, <span class="number">4</span>, <span class="number">6</span>, <span class="number">7</span>, <span class="number">13</span>,
|
|
<span class="number">3</span>, <span class="number">7</span>, <span class="number">3</span>, <span class="number">3</span>, <span class="number">1</span>, <span class="number">3</span>, <span class="number">3</span>, <span class="number">10</span>, <span class="number">3</span>, <span class="number">1</span>, <span class="number">5</span>, <span class="number">2</span>, <span class="number">0</span>, <span class="number">1</span>, <span class="number">2</span>,
|
|
<span class="number">16</span>, <span class="number">3</span>, <span class="number">5</span>, <span class="number">1</span>, <span class="number">6</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">2</span>, <span class="number">58</span>, <span class="number">20</span>, <span class="number">3</span>, <span class="number">14</span>, <span class="number">12</span>, <span class="number">2</span>, <span class="number">1</span>, <span class="number">3</span>, <span class="number">16</span>, <span class="number">3</span>, <span class="number">5</span>,
|
|
<span class="number">8</span>, <span class="number">3</span>, <span class="number">1</span>, <span class="number">8</span>, <span class="number">6</span>, <span class="number">17</span>, <span class="number">6</span>, <span class="number">5</span>, <span class="number">3</span>, <span class="number">8</span>, <span class="number">6</span>, <span class="number">13</span>, <span class="number">175</span>,
|
|
];
|
|
TABLE[byte <span class="kw">as </span>usize]
|
|
}
|
|
}
|
|
<span class="comment">// Create a new finder with the custom heuristic.
|
|
</span><span class="kw">let </span>finder = FinderBuilder::new()
|
|
.build_forward_with_ranker(Binary, <span class="string">b"\x00\x00\xdd\xdd"</span>);
|
|
<span class="comment">// Find needle with custom heuristic.
|
|
</span><span class="macro">assert!</span>(finder.find(<span class="string">b"\x00\x00\x00\xdd\xdd"</span>).is_some());</code></pre></div></div></details><h2 id="required-methods" class="section-header">Required Methods<a href="#required-methods" class="anchor">§</a></h2><div class="methods"><details class="toggle method-toggle" open><summary><section id="tymethod.rank" class="method"><a class="src rightside" href="../../../../src/memchr/arch/all/packedpair/mod.rs.html#317">Source</a><h4 class="code-header">fn <a href="#tymethod.rank" class="fn">rank</a>(&self, byte: <a class="primitive" href="https://doc.rust-lang.org/1.93.1/std/primitive.u8.html">u8</a>) -> <a class="primitive" href="https://doc.rust-lang.org/1.93.1/std/primitive.u8.html">u8</a></h4></section></summary><div class="docblock"><p>Return the heuristic frequency rank of the given byte. A lower rank
|
|
means the byte is believed to occur less frequently in the haystack.</p>
|
|
<p>Some uses of this heuristic may treat arbitrary absolute rank values as
|
|
significant. For example, an implementation detail in this crate may
|
|
determine that heuristic prefilters are inappropriate if every byte in
|
|
the needle has a “high” rank.</p>
|
|
</div></details></div><h2 id="foreign-impls" class="section-header">Implementations on Foreign Types<a href="#foreign-impls" class="anchor">§</a></h2><details class="toggle implementors-toggle"><summary><section id="impl-HeuristicFrequencyRank-for-%26R" class="impl"><a class="src rightside" href="../../../../src/memchr/arch/all/packedpair/mod.rs.html#331-338">Source</a><a href="#impl-HeuristicFrequencyRank-for-%26R" class="anchor">§</a><h3 class="code-header">impl<'a, R> <a class="trait" href="trait.HeuristicFrequencyRank.html" title="trait memchr::arch::all::packedpair::HeuristicFrequencyRank">HeuristicFrequencyRank</a> for <a class="primitive" href="https://doc.rust-lang.org/1.93.1/std/primitive.reference.html">&'a R</a><div class="where">where
|
|
R: <a class="trait" href="trait.HeuristicFrequencyRank.html" title="trait memchr::arch::all::packedpair::HeuristicFrequencyRank">HeuristicFrequencyRank</a>,</div></h3><div class="docblock"><p>This permits passing any implementation of <code>HeuristicFrequencyRank</code> as a
|
|
borrowed version of itself.</p>
|
|
</div></section></summary><div class="impl-items"><section id="method.rank" class="method trait-impl"><a class="src rightside" href="../../../../src/memchr/arch/all/packedpair/mod.rs.html#335-337">Source</a><a href="#method.rank" class="anchor">§</a><h4 class="code-header">fn <a href="#tymethod.rank" class="fn">rank</a>(&self, byte: <a class="primitive" href="https://doc.rust-lang.org/1.93.1/std/primitive.u8.html">u8</a>) -> <a class="primitive" href="https://doc.rust-lang.org/1.93.1/std/primitive.u8.html">u8</a></h4></section></div></details><h2 id="implementors" class="section-header">Implementors<a href="#implementors" class="anchor">§</a></h2><div id="implementors-list"></div><script src="../../../../trait.impl/memchr/arch/all/packedpair/trait.HeuristicFrequencyRank.js" async></script></section></div></main></body></html> |