Sample Set Management#

SampleSet#

class ReadOnlySampleSet#

ReadOnlySampleSet is a read-only database to store list of samples. User can query the samples by using the iterator or the index.

See also

SampleSet

Subclassed by visionflow::data::SampleSet

Public Functions

bool exists(uint32_t id) const#

Check If the ID is exists in the sample set.

Parameters:: id – sample id
Returns:: true
Returns:: false

size_t size() const#

Get sample count of the sample set.

Warning

This operation is expensive as it needs to iterate all samples, so it is not recommended to use it in a loop or in a performance critical path.

Returns:: size_t sample count.

bool empty() const#

Check if the SampleSet is empty.

Throws:: visionflow::excepts::UnknownError – Throws if the first iterator of Rocksdb points to an invalid Sample.
Returns:: true
Returns:: false

LazySample at(uint32_t id) const#

Get LazySample corresponding to the ID.

Parameters:: id – sample id
Throws:: visionflow::excepts::SampleNotFound – Sample with the input id does exists in the sample set.
Returns:: LazySample

SampleDescriptor sample_descriptor(uint32_t id) const#

Get Sample descriptor.

Parameters:: id – The sample id
Returns:: SampleDescriptor

SampleSetIterator begin() const#

Get sample set iterator handle of pointing to the first sample.

Returns:: SampleSetIterator pointing to the first sample.

SampleSetIterator find(uint32_t id) const#

Get sample set iterator handle of pointing to the corresponding sample of the ID.

Note

Return end() if the sample id not exist in SampleSet.

Parameters:: id – sample id.
Returns:: SampleSetIterator pointing to the corresponding sample of the ID.

SampleSetIterator end() const#

Get the next SampleSetIterator pointing to the last Property.

Returns:: The next SampleSetIterator pointing to the last property.

ReadOnlyPropertySet readonly_property_set(const ToolNodeId &property_name) const#

Get read only property set handle.

Note

The ReadOnlyPropertySet handle is dependent on the sample set, so it should not be used after the sample set is closed.

Throws:: visionflow::excepts::SampleNotFound – The input property name is not found.
Returns:: ReadOnlyPropertySet The read only property set.

std::string property_type(const ToolNodeId &property_name) const#

Get the property type of the given property name.

Parameters:: property_name – The property name.
Returns:: NO_DISCARD The type of the property.

ReadOnlySampleSetView filter(const std::string &filter_script)#

Filter on ReadOnlySampleSet.

Parameters:: filter_script – Python filter script. ReadOnlySampleSetView::ReadOnlySampleSetView(const ReadOnlySampleSet&sample_set, const std::string &filter_script)
Returns:: ReadOnlySampleSetView

Sample create_empty_sample() const#

Create a empty sample object. this function only create an empty sample template and will not change the database.

Returns:: Empty sample object.

using visionflow::ReadOnlySampleSet = data::ReadOnlySampleSet #

class SampleSet : public visionflow::data::ReadOnlySampleSet #

SampleSet is a database to store list of samples. User can add, remove or modify the samples in the sample set.

See also

ReadOnlySampleSet

Public Functions

uint32_t add_empty_sample()#

Create a empty sample and add it to the sample set, return the ID of the sample and the sample handle.

Throws:: visionflow::excepts::RocksDBError – Get or Put counter_max_id failed.
Returns:: uint32_t The Sample ID and sample handle.

uint32_t add(const ISample &data)#

Store the input sample into the sample set and return the ID.

Parameters:

data – sample

Throws:

visionflow::excepts::PropertyTypeMismatch – Some property names and types in the input sample is not equal to the sample set.
visionflow::excepts::RocksDBError – Failed to get new sample id.

Returns:

uint32_t sample id.

void update(uint32_t id, const ISample &data)#

Update all data of the ID in the sample set to the input sample, the existing property will be overwritten.

Note

If some properties in the input sample are empty, they will not be updated.

Throws:

visionflow::excepts::SampleNotFound – Sample with the input id does not exist in the sample set.
visionflow::excepts::PropertyTypeMismatch – Some property names and types in the input sample is not equal to the sample set.

Parameters:

id – sample id.
data –

void update(uint32_t id, const SampleDescriptor &data)#

Update the descriptor of the sample with ID in the sample set, the existing sample descriptor will be overwritten.

Throws:

visionflow::excepts::SampleNotFound – Sample with the input id does not exist in the sample set.

Parameters:

id – sample id.
data –

void update(const SampleSetIterator &iter, const ISample &data)#

Update all data of the iterator in the sample set to the input sample, the existing property will be overwritten.

Throws:

visionflow::excepts::InvalidIterator – The input iterator is invalid.

Parameters:

iter – sample set iterator handle.
data –

void update(const SampleSetIterator &iter, const SampleDescriptor &data)#

Update the descriptor of the sample with ID in the sample set, the existing sample descriptor will be overwritten.

Throws:

visionflow::excepts::InvalidIterator – The input iterator is invalid.

Parameters:

iter – sample set iterator handle.
data –

void erase(uint32_t id)#

Delete all data of this ID in the sample set.

Throws:

visionflow::excepts::SampleNotFound – Sample with the input id does not exist in the sample set.
visionflow::excepts::RocksDBError – delete some one property failed.

Parameters:

id – sample id.

SampleSetIterator erase(const SampleSetIterator &iter)#

Delete all data of this iterator in the sample set, and return the iterator pointing to the next sample.

Note

If the input iterator pointing to last sample, return iterator is invalid (ret_iter.valid() = false).

Parameters:: iter – sample set iterator handle.
Throws:: visionflow::excepts::InvalidIterator – The input iterator is invalid.
Returns:: SampleSetIterator

PropertySet property_set(const ToolNodeId &property_name)#

Get property set handle.

Note

The PropertySet handle is dependent on the sample set, so it should not be used after the sample set is closed.

Throws:: visionflow::excepts::SampleNotFound – The input property name is not found.
Returns:: PropertySet The property set handle.

SampleWriteBatch create_write_batch() const#

Create an SampleWriteBatch object that caches SampleSet writes.

Returns:: NO_DISCARD

void write_batch(const SampleWriteBatch &write_batch)#

Run write batch.

Parameters:

write_batch – SampleWriteBatch.

Throws:

visionflow::excepts::PropertyTypeMismatch – Some property names and types in the input sample is not equal to the sample set.
visionflow::excepts::SampleNotFound – If there is a sample id that does not exist.
visionflow::excepts::RocksDBError – If the database access operation fails.

using visionflow::SampleSet = data::SampleSet #

class SampleSetIterator#

SampleSetIterator is a iterator to iterate all samples in the sample set.

See also

ReadOnlySampleSet

Public Functions

SampleSetIterator &operator++()#

iterator pointing to next sample.

Attention: The iterator must be valid.

Returns:: SampleSetIterator&

SampleSetIterator &operator--()#

iterator pointing to prev sample.

Attention: The iterator must be valid.

Returns:: SampleSetIterator&

SampleSetIterator operator++(int)#

return separate copy of the current iterator, and iterator pointing to next sample.

Attention: The iterator must be valid.

Returns:: SampleSetIterator separate copy of the current iterator.

SampleSetIterator operator--(int)#

return separate copy of the current iterator, and iterator pointing to prev sample.

Attention: The iterator must be valid.

Returns:: SampleSetIterator separate copy of the current iterator.

void seek_to_first()#: Set Iterator to the first sample.

void seek_to(uint32_t id)#

Set Iterator to the sample with the given id.

Parameters:: id – The id of the sample.

void seek_to_last()#: Set Iterator to the last sample.

bool operator!=(const SampleSetIterator &other) const#

Judge whether two iterators pointing to the different sample.

Attention: The iterator must be valid.

Parameters:: other –
Returns:: true
Returns:: false

bool operator==(const SampleSetIterator &other) const#

Judge whether two iterators pointing to the same sample.

Attention: The iterator must be valid.

Parameters:: other –
Returns:: true
Returns:: false

std::pair<uint32_t, LazySample> operator*() const#

Get LazySample of corresponding to the current iterator.

Attention: The iterator must be valid.

Returns:: std::pair<uint32_t, LazySample>

bool valid() const#

Check if the iterator is valid.

Attention: Make sure the iterator is valid before calling the operation of the iterator.

Returns:: true
Returns:: false

uint32_t key() const#

Get the ID of the sample pointing to by the current iterator.

Attention: The iterator must be valid.

Throws:: visionflow::excepts::InvalidIterator – The iterator is invalid.
Returns:: uint32_t sample id of current iterator.

LazySample value() const#

Get LazySample of corresponding to the current iterator.

Attention: The iterator must be valid.

Throws:: visionflow::excepts::InvalidIterator – The iterator is invalid.
Returns:: LazySample

SampleDescriptor descriptor() const#

Get SampleDescriptor object of this sample.

Returns:: SampleDescriptor The sample descriptor.

using visionflow::SampleSetIterator = data::SampleSetIterator #

PropertySet#

class ReadOnlyPropertySet : public visionflow::data::IReadOnlyPropertySet #

Read-only property set.

Subclassed by visionflow::data::PropertySet

Public Functions

virtual bool sample_exists(uint32_t id) const override#

Check If the ID exists in the property set.

Parameters:: id – sample id
Returns:: true
Returns:: false

virtual bool data_exists(uint32_t id) const override#

Check If the property data to the ID exists in the property set.

Parameters:: id – sample id
Returns:: true
Returns:: false

virtual size_t sample_size() const override#

Get sample count of the sample set.

See also

visionflow::SampleSet::size()

Warning

This operation is expensive as it needs to iterate all samples, so it is not recommended to use it in a loop or in a performance critical path.

Returns:: size_t sample count.

virtual bool sample_empty() const override#

Check if the sample set is empty.

See also

visionflow::SampleSet::empty()

Throws:: visionflow::excepts::UnknownError – Throws if the first iterator of Rocksdb points to an invalid Sample.
Returns:: true
Returns:: false

virtual std::shared_ptr<props::IProperty> at(uint32_t id) const override#

Get property data pointer corresponding to the ID.

Throws:: visionflow::excepts::SampleNotFound – Failed to create iterator for property set, maybe the specified by input id does not exist.
Parameters:: id – sample id
Returns:: std::shared_ptr<props::IProperty> The property data specified by the input id.
Returns:: nullptr If the input id does not exist in the property set.

virtual PropertySetIterator begin() const override#

Get PropertySetIterator of pointing to the first Property.

Returns:: PropertySetIterator pointing to the first Property.

virtual PropertySetIterator find(uint32_t id) const override#

Get PropertySetIterator of pointing to the corresponding property of sample ID.

Parameters:: id – sample id
Returns:: PropertySetIterator pointing to corresponding property of the ID. or the end pointer if property specified by the id not exists.

virtual PropertySetIterator end() const override#

Get the next PropertySetIterator pointing to the last Property.

Returns:: The next PropertySetIterator pointing to the last property.

virtual int64_t last_update_time() const override#

Get last update time of the property set in utc timestamp.

Returns:: int64_t The last update time of the property set in utc timestamp.

ReadOnlyPropertySetView filter(const std::string &filter_script) const#

Get read only property set view handle.

Parameters:: filter_script – Python filter script. ReadOnlyPropertySetView
Returns:: ReadOnlyPropertySetView

std::string property_type() const#

Get the type of the property.

Returns:: NO_DISCARD Property type.

using visionflow::ReadOnlyPropertySet = data::ReadOnlyPropertySet #

class PropertySet : public visionflow::data::ReadOnlyPropertySet #

PropertySet read-write data accessor for property set.

Public Functions

void update(uint32_t id, const props::IProperty &property)#

Update property of the ID in the property set to the input property, the existing property will be overwritten.

Throws:

visionflow::excepts::RocksDBError – the property get or put error.
visionflow::excepts::SampleNotFound – Failed to seek iterator to the last property in the property set.

Parameters:

id – sample id
property –

void update(const PropertySetIterator &iter, const props::IProperty &property)#

Update property of the iterator in the property set to the input property, the existing property will be overwritten. update(uint32_t id, const props::IProperty &property)

Attention: The PropertySetIterator must be valid.

Throws:

visionflow::excepts::InvalidIterator – The input iterator is not point to a valid sample.

Parameters:

iter – PropertySetIterator from sample set that needs to be updated
property –

void erase(uint32_t id)#

Delete the property of the ID from the property set.

Throws:: visionflow::excepts::SampleNotFound – Sample with the input id does not exist in the property set.
Parameters:: id – sample id.

PropertySetIterator erase(const PropertySetIterator &iter)#

Delete the property of the PropertySetIterator from the property set, return iterator that points to the next property.

Attention: The PropertySetIterator must be valid and consistent with property set.
Attention: If iter pointing to last Property, return PropertySetIterator that is invalid.

Throws:: visionflow::excepts::InvalidIterator – The input iterator is not point to a valid sample.
Parameters:: iter – PropertySetIterator to be deleted from property set
Returns:: PropertySetIterator pointing to the next property.

PropertyWriteBatch create_write_batch() const#

Create an PropertyWriteBatch object that caches PropertySet writes.

Returns:: NO_DISCARD

void write_batch(const PropertyWriteBatch &write_batch)#

Run write batch.

Parameters:

write_batch – PropertyWriteBatch.

Throws:

visionflow::excepts::SampleNotFound – If there is a sample id that does not exist.
visionflow::excepts::RocksDBError – If the database access operation fails.

using visionflow::PropertySet = data::PropertySet #

class PropertyWriteBatch#

Cache write operations for PropertySet.

Public Functions

void update(uint32_t id, const props::IProperty &data)#

Cache an update operation.

Parameters:

id – sample id.
data –

Throws:

visionflow::excepts::PropertyTypeMismatch – The property type of the data is not equal to the required type.

void erase(uint32_t id)#

Cache an erase operation.

Parameters:: id –

using visionflow::PropertyWriteBatch = data::PropertyWriteBatch #

class SampleWriteBatch#

Public Functions

void update(uint32_t id, const ISample &data)#

Cache the update operation of a sample, and only update the valid property data in this sample.

Parameters:

id – sample id.
data –

Throws:

visionflow::excepts::PropertyTypeMismatch – The property types of the data are not equal to the required types.
visionflow::excepts::SampleNotFound – The sample has been cached as erase.

void update(uint32_t id, const SampleDescriptor &data)#

Cache a sample descriptor update operation.

Parameters:

id – sample id.
data –

Throws:

visionflow::excepts::SampleNotFound – The sample has been cached as erase.

void update(uint32_t id, const ToolNodeId &property_name, const std::shared_ptr<props::IProperty> &data)#

Cache one of the property data of the specified sample.

Parameters:

id – sample id.
property_name – The property name.
data –

Throws:

visionflow::excepts::DataNotFound – The input property name is invalid.
visionflow::excepts::PropertyTypeMismatch – The property type of the data is not equal to the required type.
visionflow::excepts::SampleNotFound – The sample has been cached as erase.

void erase(uint32_t id)#

Cache the erase operation of an entire sample.

Parameters:: id – sample id.

void erase(uint32_t id, const ToolNodeId &property_name)#

Cache the erase operation of one of the property data of the specified sample.

Parameters:

id – sample id.
property_name – The property name.

Throws:

visionflow::excepts::DataNotFound – The input property name is invalid.

using visionflow::SampleWriteBatch = data::SampleWriteBatch #

class PropertySetIterator#

Abstract iterator for IPropertySet.

Subclassed by visionflow::data::PropertySetIterTPL< PropertyType >

Public Functions

inline PropertySetIterator &operator++()#

iterator pointing to next property.

Attention: The iterator must be valid.

Returns:: PropertySetIterator&

inline PropertySetIterator &operator--()#

iterator pointing to prev property.

Attention: The iterator must be valid.

Returns:: PropertySetIterator&

inline PropertySetIterator operator++(int)#

return separate copy of the current iterator, and iterator pointing to next property.

Attention: The iterator must be valid.

Returns:: PropertySetIterator separate copy of the current iterator.

inline PropertySetIterator operator--(int)#

return separate copy of the current iterator, and iterator pointing to prev Property.

Attention: The iterator must be valid.

Returns:: PropertySetIterator

inline std::pair<uint32_t, std::shared_ptr<props::IProperty>> operator*() const#

Get property of corresponding to the current iterator.

Returns:: std::pair<uint32_t, std::shared_ptr<props::IProperty>> The sample id and property.

inline bool operator==(const PropertySetIterator &other) const#

Judge whether two iterators pointing to the same property.

Note

Two invalid iterators are always not equal to each other.

Parameters:: other – The other iterator.
Returns:: true if two iterators pointing to the same property.
Returns:: false if two iterators pointing to the different property, or one of the iterator is invalid.

inline bool operator!=(const PropertySetIterator &other) const#

Judge whether two iterators pointing to the different property.

Note

Two invalid iterators are always not equal to each other.

Parameters:: other – The other iterator.
Returns:: true if two iterators pointing to the different property. or one of the iterator is invalid.
Returns:: false if two iterators pointing to the same property.

inline bool valid() const#

Check if the iterator is valid.

Attention: Make sure the iterator is valid before calling the operation of the iterator.

Returns:: true if the iterator is valid.
Returns:: false if the iterator is invalid.

inline uint32_t key() const#

Get the ID of the property pointing to by the current iterator.

Attention: The iterator must be valid.

Throws:: visionflow::excepts::InvalidIterator – The iterator is invalid.
Returns:: uint32_t sample id of current iterator.

inline std::shared_ptr<props::IProperty> value() const#

Get Property of corresponding to the current iterator.

Throws:: visionflow::excepts::InvalidIterator – The iterator is invalid.
Returns:: std::shared_ptr<props::IProperty> Property data of current iterator. It will be nullptr if the sample exists but the property data is not set.

using visionflow::PropertySetIterator = data::PropertySetIterator #

template<typename PropertyType> class PropertySetIterTPL : public visionflow::data::PropertySetIterator #

Template class based on PropertySet Iterator.

Template Parameters:: PropertyType – The type of property to be returned by the iterator.

Public Functions

inline std::shared_ptr<PropertyType> value() const#

Get the property with type PropertyType from the iterator.

Returns:: std::shared_ptr<PropertyType> The property.

template<typename PropertyType> class ReadOnlyPropertySetTPL#

Public Types

using Iterator = PropertySetIterTPL<PropertyType>#: the iterator type.

Public Functions

inline bool sample_exists(uint32_t id) const#

Check If the ID exists in the property set.

Parameters:: id – sample id
Returns:: true
Returns:: false

inline bool data_exists(uint32_t id) const#

Check If the property data to the ID exists in the property set.

Parameters:: id – sample id
Returns:: true
Returns:: false

inline size_t sample_size() const#

Get sample count of the sample set.

See also

visionflow::SampleSet::size()

Warning

This operation is expensive as it needs to iterate all samples, so it is not recommended to use it in a loop or in a performance critical path.

Returns:: size_t sample count.

inline bool sample_empty() const#

Check if the sample set is empty.

See also

visionflow::SampleSet::empty()

Throws:: visionflow::excepts::UnknownError – Throws if the first iterator of Rocksdb points to an invalid Sample.
Returns:: true
Returns:: false

inline std::shared_ptr<PropertyType> at(uint32_t id) const#

Get property with type PropertyType from the property set.

See also

visionflow::data::ReadOnlyPropertySet::get()

Parameters:: id – sample id
Returns:: std::shared_ptr<PropertyType> The property.

inline Iterator begin() const#

Get PropertySetIterator of pointing to the first Property.

Returns:: PropertySetIterator pointing to the first Property.

inline Iterator find(uint32_t id) const#

Get property data pointer corresponding to the ID.

Throws:: visionflow::excepts::SampleNotFound – Failed to create iterator for property set, maybe the specified by input id does not exist.
Parameters:: id – sample id
Returns:: std::shared_ptr<props::IProperty> The property data specified by the input id.
Returns:: nullptr If the input id does not exist in the property set.

inline Iterator end() const#

Get the next PropertySetIterator pointing to the last Property.

Returns:: The next PropertySetIterator pointing to the last property.

inline int64_t last_update_time() const#

Get last update time of the property set in utc timestamp.

Returns:: int64_t The last update time of the property set in utc timestamp.

Abstract Interfaces#

class IReadOnlyPropertySet#

IReadOnlyPropertySet abstract interface. A PropertySet is a data structure used to manage list of properties. The properties in a PropertySet have the same data type and correspond to the data of the same node in the stream graph of different samples.

Subclassed by visionflow::data::ReadOnlyPropertySet, visionflow::data::ReadOnlyPropertySetView

Public Functions

virtual bool sample_exists(uint32_t id) const = 0#

Check If the ID exists in the property set.

Parameters:: id – sample id
Returns:: true
Returns:: false

virtual bool data_exists(uint32_t id) const = 0#

Check If the property data to the ID exists in the property set.

Parameters:: id – sample id
Returns:: true
Returns:: false

virtual size_t sample_size() const = 0#

Get sample count of the sample set.

See also

visionflow::SampleSet::size()

Warning

This operation is expensive as it needs to iterate all samples, so it is not recommended to use it in a loop or in a performance critical path.

Returns:: size_t sample count.

virtual bool sample_empty() const = 0#

Check if the sample set is empty.

Throws:: visionflow::excepts::UnknownError – Throws if the first iterator of Rocksdb points to an invalid Sample.
Returns:: true
Returns:: false

virtual std::shared_ptr<props::IProperty> at(uint32_t id) const = 0#

Get property data pointer corresponding to the ID.

Throws:: visionflow::excepts::SampleNotFound – Failed to create iterator for property set, maybe the specified by input id does not exist.
Parameters:: id – sample id
Returns:: std::shared_ptr<props::IProperty> The property data specified by the input id.
Returns:: nullptr If the input id does not exist in the property set.

virtual PropertySetIterator begin() const = 0#

Get PropertySetIterator of pointing to the first Property.

Throws:: visionflow::excepts::SampleNotFound – Failed to seek iterator to the first property in the property set. maybe the property set is empty.
Returns:: PropertySetIterator pointing to the first Property.

virtual PropertySetIterator find(uint32_t id) const = 0#

Find PropertySetIterator of pointing to the corresponding property of sample ID.

Throws:: visionflow::excepts::SampleNotFound – Failed to create iterator for property set, maybe the sample specified by input id does not exist.”,
Parameters:: id – sample id
Returns:: PropertySetIterator pointing to corresponding property of the ID.

virtual PropertySetIterator end() const = 0#

Get the next PropertySetIterator pointing to the last Property.

Returns:: The next PropertySetIterator pointing to the last property.

inline virtual int64_t last_update_time() const#

Get last update time of the property set in utc timestamp.

Returns:: int64_t The last update time of the property set in utc timestamp.

class IPropertySetIterator#

Abstract iterator for IPropertySet.

Public Functions

virtual IPropertySetIterator *clone() = 0#

Clone to a new iterator.

Returns:: IPropertySetIterator*

virtual IPropertySetIterator &operator++() = 0#

iterator pointing to next property.

Attention: The iterator must be valid.

Returns:: IPropertySetIterator&

virtual IPropertySetIterator &operator--() = 0#

iterator pointing to prev property.

Attention: The iterator must be valid.

Returns:: IPropertySetIterator&

virtual IPropertySetIterator *operator++(int) = 0#

return separate copy of the current iterator, and iterator pointing to next property.

Attention: The iterator must be valid.

Returns:: IPropertySetIterator separate copy of the current iterator.

virtual IPropertySetIterator *operator--(int) = 0#

return separate copy of the current iterator, and iterator pointing to prev Property.

Attention: The iterator must be valid.

Returns:: IPropertySetIterator

inline virtual std::pair<uint32_t, std::shared_ptr<props::IProperty>> operator*() const#

Get property of corresponding to the current iterator.

Returns:: std::pair<uint32_t, std::shared_ptr<props::IProperty>> The sample id and property.

virtual bool operator==(const IPropertySetIterator &other) const = 0#

Judge whether two iterators pointing to the same property.

Parameters:: other – The other iterator.
Returns:: true if two iterators pointing to the same property.
Returns:: false if two iterators pointing to the different property.

inline bool operator!=(const IPropertySetIterator &other) const#

Judge whether two iterators pointing to the different property.

Parameters:: other – The other iterator.
Returns:: true if two iterators pointing to the different property.
Returns:: false if two iterators pointing to the same property.

virtual bool valid() const = 0#

Check if the iterator is valid.

Attention: Make sure the iterator is valid before calling the operation of the iterator.

Returns:: true if the iterator is valid.
Returns:: false if the iterator is invalid.

virtual uint32_t key() const = 0#

Get the ID of the property pointing to by the current iterator.

Attention: The iterator must be valid.

Throws:: visionflow::excepts::InvalidIterator – The iterator is invalid.
Returns:: uint32_t sample id of current iterator.

virtual std::shared_ptr<props::IProperty> value() const = 0#

Get Property of corresponding to the current iterator.

Throws:: visionflow::excepts::InvalidIterator – The iterator is invalid.
Returns:: std::shared_ptr<props::IProperty> Property data of current iterator. It will be nullptr if the sample exists but the property data is not set.

virtual Buffer raw_data() const = 0#

Get raw binary data.

Returns:: NO_DISCARD

ReadOnlyPropertySetView#

class ReadOnlyPropertySetView : public visionflow::data::IReadOnlyPropertySet #

Public Functions

virtual bool sample_exists(uint32_t id) const override#

Check If the ID exists in the property set.

Parameters:: id – sample id
Returns:: true
Returns:: false

virtual bool data_exists(uint32_t id) const override#

Check If the property data to the ID exists in the property set.

Note

Even if the data exists, it does not mean that it meets the filtering criteria. No filtering conditions will be checked here.

Parameters:: id – sample id
Returns:: true
Returns:: false

virtual size_t sample_size() const override#

Get sample count of the sample set.

See also

visionflow::SampleSet::size()

Warning

This operation is expensive as it needs to iterate all samples, so it is not recommended to use it in a loop or in a performance critical path.

Returns:: size_t sample count.

virtual bool sample_empty() const override#

Check if the sample set is empty.

See also

visionflow::SampleSet::empty()

Throws:: visionflow::excepts::UnknownError – Throws if the first iterator of Rocksdb points to an invalid Sample.
Returns:: true
Returns:: false

virtual std::shared_ptr<props::IProperty> at(uint32_t id) const override#

Get property data pointer corresponding to the ID.

Throws:: visionflow::excepts::SampleNotFound – Failed to create iterator for property set, maybe the specified by input id does not exist.
Parameters:: id – sample id
Returns:: std::shared_ptr<props::IProperty> The property data specified by the input id.
Returns:: nullptr It will be nullptr if the iterator is invalid or does’t meet the filter criteria.

virtual PropertySetIterator begin() const override#

Get PropertySetIterator of pointing to the first Property.

Throws:: visionflow::excepts::SampleNotFound – Failed to seek iterator to the first property in the property set. maybe the property set is empty.
Returns:: PropertySetIterator pointing to the first Property.

virtual PropertySetIterator find(uint32_t id) const override#

Get PropertySetIterator of pointing to the corresponding property of sample ID.

Throws:: visionflow::excepts::SampleNotFound – Failed to create iterator for property set and end pointer not exists, end() for more details.
Parameters:: id – sample id
Returns:: PropertySetIterator pointing to corresponding property of the ID. or the end pointer if property specified by the id not exists.

virtual PropertySetIterator end() const override#

Get the next PropertySetIterator pointing to the last Property.

Returns:: The next PropertySetIterator pointing to the last property.

virtual int64_t last_update_time() const override#

Get last update time of the property set in utc timestamp.

Returns:: int64_t The last update time of the property set in utc timestamp.

ReadOnlyPropertySetView filter(const std::string &filter_script) const#

Get read only property set view handle.

Parameters:: filter_script – Python filter script. ReadOnlyPropertySetView
Returns:: ReadOnlyPropertySetView

ReadOnlyPropertySetView &set_filter_ids(const std::vector<uint32_t> &filter_ids)#

The data in filter ids will be filtered out.

Parameters:: filter_ids –
Returns:: ReadOnlyPropertySetView&

ReadOnlyPropertySetView &set_filter_script(const std::string &filter_script)#

Set the filter script.

Parameters:: filter_script –
Returns:: ReadOnlyPropertySetView&

using visionflow::ReadOnlyPropertySetView = data::ReadOnlyPropertySetView #

ReadOnlySampleSetView#

class ReadOnlySampleSetView#

ReadOnlySampleSet filtering engine.

Public Functions

explicit ReadOnlySampleSetView(const ReadOnlySampleSet &sample_set, const std::string &filter_script)#

It will not filter the datasets during construction. 1.If you want to filter all datasets at one time and get the qualified dataset ids, please call the ids(). 2.If your data set is very large and want to get filtered data one by one, please call exists(uint32_t id) const, at(uint32_t id) const, or iterator.

Note

1.Note that it does not cache your previous filtering results, which means that if your data set changes outside, you don’t need to make any changes. When you call the interface again, you will get the filtering results of the latest data set. This also means that every call to ids() will perform filtering on the entire dataset, please be careful when calling this interface. 2.Note that since no filtering will be performed here, please confirm that the filtering rules can correctly run the filtering of the target sample.

Parameters:

sample_set – The SampleSet to be filtered.
filter_script – Python filter script. eg:
```
R"(
from visionflow.property import *
def vflow_filter(data):
    id1 = ToolNodeId("Tool_0", "prop_1")
    id2 = ToolNodeId("Tool_0", "prop_2")
    if not(data.exist_property_data(id1) and data.exist_property_data(id2)):
        return False
    return data.get(id1).get_y() == 1 or data[id2].get_w() == 2)"
```
Have to be aware of is: 1.It needs to be a complete filter script that conforms to python syntax, and pay attention to indentation. 2.The name of the function must be ‘vflow_filter’, and there can be only one parameter with any name. 3.As for the function body, write your filter function, and finally return ‘true’ or ‘false’.

Throws:

visionflow::excepts::PythonSyntaxError – If the filter script does not conform to the python syntax rules, or the module import statement is not a valid python module import statement.

std::vector<uint32_t> ids() const#

Filter the entire dataset and return the dataset ids that match the filter script.

Note

Every time this interface is called, the entire data set will be filtered, please call with caution.

Throws:: visionflow::excepts::PythonSyntaxError – The filtering script cannot perform Sample filtering correctly.
Returns:: NO_DISCARD std::vector<uint32_t> All data ids that match the filter script.

bool exists(uint32_t id) const#

Check if the ID is exists in the ReadOnlySampleSetView.

Parameters:: id – sample id
Throws:: visionflow::excepts::PythonSyntaxError – The filtering script cannot perform Sample filtering correctly.
Returns:: true
Returns:: false

LazySample at(uint32_t id) const#

Get sample corresponding to the ID.

Parameters:

id – The id of the data.

Throws:

visionflow::excepts::SampleNotFound – Sample with the input id does exists in the sample set, or does not satisfy the filter script.
visionflow::excepts::PythonSyntaxError – The filtering script cannot perform Sample filtering correctly.

Returns:

NO_DISCARD

ReadOnlySampleSetViewIterator begin() const#

Get sample set iterator handle of pointing to the first sample that satisfies the filter script.

Throws:

visionflow::excepts::SampleNotFound – Failed to seek iterator to the first sample in the sample set. maybe the sample set is empty.
visionflow::excepts::PythonSyntaxError – The filtering script cannot perform Sample filtering correctly.

Returns:

ReadOnlySampleSetViewIterator pointing to the first sample that satisfies the filter script.

ReadOnlySampleSetViewIterator end() const#

Get the next ReadOnlySampleSetViewIterator pointing to the last Sample.

Returns:: The next ReadOnlySampleSetViewIterator pointing to the last Sample.

using visionflow::ReadOnlySampleSetView = data::ReadOnlySampleSetView #

class ReadOnlySampleSetViewIterator#

Public Functions

ReadOnlySampleSetViewIterator &operator++()#

Iterator pointing to the next sample matching the filter script.

Attention: The iterator must be valid.

Throws:: visionflow::excepts::PythonSyntaxError – The filtering script cannot perform sample filtering correctly.
Returns:: ReadOnlySampleSetViewIterator&

ReadOnlySampleSetViewIterator &operator--()#

Iterator pointing to the prev sample matching the filter script.

Attention: The iterator must be valid.

Throws:: visionflow::excepts::PythonSyntaxError – The filtering script cannot perform sample filtering correctly.
Returns:: ReadOnlySampleSetViewIterator&

ReadOnlySampleSetViewIterator operator++(int)#

Return separate copy of the current iterator, and iterator pointing to the next sample matching the filter script.

Attention: The iterator must be valid.

Throws:: visionflow::excepts::PythonSyntaxError – The filtering script cannot perform sample filtering correctly.
Returns:: ReadOnlySampleSetViewIterator Separate copy of the current iterator.

ReadOnlySampleSetViewIterator operator--(int)#

Return separate copy of the current iterator, and iterator pointing to the prev sample matching the filter script.

Attention: The iterator must be valid.

Throws:: visionflow::excepts::PythonSyntaxError – The filtering script cannot perform sample filtering correctly.
Returns:: ReadOnlyPropertySetViewIterator Separate copy of the current iterator.

std::pair<uint32_t, LazySample> operator*() const#

Get sample of corresponding to the current iterator.

Returns:: std::pair<uint32_t, LazySample>

bool operator==(const ReadOnlySampleSetViewIterator &other) const#

Judge whether two iterators point to the same sample that satisfies the same filter script.

Note

Two invalid iterators are always not equal to each other.

Parameters:: other –
Returns:: true
Returns:: false

bool valid() const#

Check if the iterator is valid.

Attention: Make sure the iterator is valid before calling the operation of the iterator.

Returns:: true
Returns:: false

uint32_t key() const#

Get the ID of the sample pointing to by the current iterator.

Attention: The iterator must be valid.

Throws:: visionflow::excepts::InvalidIterator – The iterator is invalid.
Returns:: uint32_t sample id of current iterator.

LazySample value() const#

Get LazySample of corresponding to the current iterator.

Attention: The iterator must be valid.

Throws:: visionflow::excepts::InvalidIterator – The iterator is invalid.
Returns:: LazySample

using visionflow::ReadOnlySampleSetViewIterator = data::ReadOnlySampleSetViewIterator #