pub enum DatasetKind {
WikiText2,
C4,
C4Wiki,
Ptb,
PtbNew,
C4New,
LocalFile(PathBuf),
}Expand description
Supported training and perplexity evaluation datasets.
Variants§
WikiText2
WikiText-2 raw (Salesforce/wikitext / wikitext-2-raw-v1).
C4
AllenAI C4 English (allenai/c4).
C4Wiki
Combined C4 + WikiText-2 training corpus (c4_wiki).
Ptb
Penn Treebank validation split (ptb).
PtbNew
Penn Treebank test split (ptb_new).
C4New
AllenAI C4 contiguous slice (c4_new).
LocalFile(PathBuf)
Local .txt, .jsonl, or .jsonl.gz corpus file.
Implementations§
Source§impl DatasetKind
impl DatasetKind
Sourcepub fn cache_name(&self) -> &str
pub fn cache_name(&self) -> &str
Canonical cache directory name matching upstream utils/datautils.py.
Trait Implementations§
Source§impl Clone for DatasetKind
impl Clone for DatasetKind
Source§impl Debug for DatasetKind
impl Debug for DatasetKind
Source§impl Display for DatasetKind
impl Display for DatasetKind
impl Eq for DatasetKind
Source§impl FromStr for DatasetKind
impl FromStr for DatasetKind
Source§impl PartialEq for DatasetKind
impl PartialEq for DatasetKind
impl StructuralPartialEq for DatasetKind
Auto Trait Implementations§
impl Freeze for DatasetKind
impl RefUnwindSafe for DatasetKind
impl Send for DatasetKind
impl Sync for DatasetKind
impl Unpin for DatasetKind
impl UnsafeUnpin for DatasetKind
impl UnwindSafe for DatasetKind
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
Checks if this value is equivalent to the given key. Read more
§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
Compare self to
key and return true if they are equal.impl<T> ErasedDestructor for Twhere
T: 'static,
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
impl<T, U> Imply<T> for U
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more§impl<T> Pointable for T
impl<T> Pointable for T
§impl<T> PolicyExt for Twhere
T: ?Sized,
impl<T> PolicyExt for Twhere
T: ?Sized,
impl<T> Read<Exclusive, BecauseExclusive> for Twhere
T: ?Sized,
impl<E> ResultError for E
impl<T> ResultType for T
§impl<T> ToCompactString for Twhere
T: Display,
impl<T> ToCompactString for Twhere
T: Display,
§fn try_to_compact_string(&self) -> Result<CompactString, ToCompactStringError>
fn try_to_compact_string(&self) -> Result<CompactString, ToCompactStringError>
Fallible version of [
ToCompactString::to_compact_string()] Read more§fn to_compact_string(&self) -> CompactString
fn to_compact_string(&self) -> CompactString
Converts the given value to a [
CompactString]. Read more