Catalogue window¶
CatalogueWindow is important, it helps users to manually validate and correct some mistakes made by the automatic steps (filtering, threshold, feature extraction, clustering, …)
All views are linked, this means that some actions interact with other views.
All views can be customised by settings (often with a double in the view).
Here some detail on each view.
Note
A lot of efforts have been put in making this UI as smooth as possible but for big datasets (>100 channels) the CatalogueWindow can be slow for some actions because it trigs a full refresh on other views and no computation can be done (re compute centroid for instance). So be patient and smart.
Note
For some manual actions on catalogue, CatalogueWindow can suddenly crash. While this is annoying, you should not lose any data. Just open again the same dataset and you should be in a previous situation.
In cases of crashes please send an issue on github https://github.com/tridesclous/tridesclous/issues It takes only minutes and helps a lot to make tridesclous more stable. Please copy/paste the error message in the console in the issue and describe very briefly the actions that triggered the crash.
Toolbar on the left¶
You can:
- Make catalogue for Peeler: this construct all waveform centroid + first and second derivative + interpolated waveforms. This is needed by the Peeler. This initial catalogue is valid for one channel group only. This is saved inside the working directory of the DataIO in the appropriate channel_group sub dir. Do this at the end when the clusters seem OK before launching the Peeler on the entire dataset.
- refresh: this reloads data from DataIO and refreshes all views. This is extremly useful when you externally in jupyter for instance change or test some methods that transform data. This avoids to restart the UI.
- New peaks: re detect peaks, you change the threshold for a new detection here.
- New waveforms: take some other (or all) waveforms with other parameters.
- Clean waveforms: detect bad (allien) waveform.
- New noise snippet: extract new noise snippet.
- New features: choose other method or parameters for features extraction.
- New cluster: choose other method or parameters for clustering.
- Compute metrics: the compute metric on clusters : similarity, ratio_similarity, silhouet, …
- Help: a button that magically telestranport you in that this current page.
- Savepoint: duplicate the working dir, that you can manually go back in the past with a funny game of folder renaming.
Peak list¶
-
class
tridesclous.gui.peaklists.PeakList(controller=None, parent=None)[source]¶ Peak List show all detected peak for the catalogue construction.
Here pintentionally peaks are not spikes already (even most of them are spikes) because supperposition of spikes are done here in catalogue in Peeler.
- Note:
- If there are to many peaks, not all of them will have a extracted waveform. This why some peak are not labeled (-10) and nb_peak != nb_wveforms sometimes.
- Peaks can belong to diffrents segment, a column indicate it. This is th full list of all peaks of all segment.
- A right click open a ontext menu: * move one or several selected spike to trash * create a new cluster with one or several spikes
- When you select one spike, this will auto zoom on Trace View, auto select the appriopriate segment and hilight the point on ND Scatetr. And vice versa.
Cluster list¶
-
class
tridesclous.gui.peaklists.ClusterPeakList(controller=None, parent=None)[source]¶ Cluster list is the central widget for actions for clusters : make them visible, merge, trash, sort, split, change color, …
- A context menu with right propose:
- Reset colors
- Show all
- Hide all
- Re-label cluster by rms: this re order cluster so that 0 have the bigger rms
- and N the lowest.
- Feature projection with all: this re compute feature projection (equivalent to left toolbar)
- Feature projection with selection: this re compute feature projection but take in account only selected usefull when you have doubt on small cluster and want a specifit PCA because variance is absord by big ones.
- Move selection to trash
- Merge selection: re label spikes in the same cluster.
- Select on peak list: a spike as selected for theses clusters.
- Tag selection as same cell: in case of burst some cell can have diffrent waveform shape leading to diferents cluster but with same ratio. If you have that do not merge clusters because the Peeler wll fail. Prefer tag 2 cluster as same cell.
- Split selection: try to split only selected cluster.
Double click on a row make invisible all others except this one.
Cluster can be visualy ordered by some criteria (rms, amplitude, nb peak, …) This is useless to explore cluster few peaks, or big amplitudes, …
- Negative labels are reserved:
- -1 is thrash
- -2 is noise snippet
- -10 unclassified (because no waveform associated)
- -9 Alien
Trace viewer¶
-
class
tridesclous.gui.traceviewer.CatalogueTraceViewer(controller=None, signal_type='processed', parent=None)[source]¶ Trace viewer allow to browser raw signal and preprocess signals.
Note that this viewer do not load the entire signals in memory but load chunk on demand from HD, that is why depending on the drive it can be quite slow. All zoom and scale factor for signals are computed on CPU and not on GPU (it is not vispy!!), so this is not the fastest veiwer but many tips help user to navigate very efficiently.
- What you can do:
- On the bottom there is a slider over time
- On the left there is a slider over channels (if nb_channel>16)
- If several segments you can switch.
- You can select manually jump to any time
- You can zoom the X (time) axis with the spinbox ot by right click with mouse.
- The mouse wheel make a glocal zoom on signal
- You can “select” manually a spike with “select” button and this will be show in ND Scatter
- The threshold is a line for each channel. This is very important to why so peak are not detected.
- Setting button: * “auto_zoom_on_select” : auto zoom when select on ndscatter on peak list * “zoom_size” in s * disable plot threshold
- Important:
- Ths “preprocessed” signal are normalized (robust Z-score) so that the noise variance is 1. So the apparent noise must be inbetween [-3, 3]
NDScatter¶
-
class
tridesclous.gui.ndscatter.NDScatter(controller=None, parent=None)[source]¶ ND Scatter a central beast for the catalogue window.
It try to mimic RGGobi viewer package.
Displaying clusters in high (more than 3!) dimensional space is difficult. To overcome this problem, some people use a combination of 2D plots of all pairs of dimensions. Others use various 3D representations. However in this two cases the view angle is seriously restricted: it is the projection of a big hyper cube (ND) onto only one face (2D or 3D) of this hyper cube. In real life, two clusters are not necessary well separated in one of this hyper-cube’s face but better in a complex hyper plane that brings into play a combination of all features/dimensions.
ND Scatter project randomly N-D space into 2-D space. Furthermore it do it dynamically. This is the random tour.
If you want to compare how 2 by 2 components are displayed in other tools you can use next face. This will combined (PC0, PC1), then (PC0, PC2), …
For very high dimention, ND Scatter is not enough, for so ND scatter is guided in feature selection with the probe geometry. In short when you set visible some label with the Cluster list this automatically activated the dimenssions that must but visible (and rodomized) and hidden. This is based on max_on_channel property of clusters. This is the auto_select_component in settings. Note that you can define a radius in micometers arround the channel of the max so that all component in a neighborhood of thsi channel will be also displayed. this is based on the PRB file that need to have the units in μm (contrary to phy where units is not important for plotting)
- Note:
- for too dense cluster you can limit the nb of dot dsiplayed. It helps a lot when density of clusters are differents. The is the role of random decimate button
- you can manually activate/deactivate componant just for to understand.
- mouse wheel will zoom both X et Y
- with right click you can draw a lasso. This will select peak inside the poly on the projected hyperplan. This is in fact the secret weappon of tridesclous. It help a lot for au zoom on trace for “strange” point ouside main clouds. It can help to understand if they are in fact superposition of several spikes or very few dense cluster.
- not all peak can be visible here only thosse wich have a waveform and so a feature. remember that not all waveforms are taken for clustering but only a subset. Getting everything is the role of the Peeler.
- This viewer take some CPU ressource since in random tour a numpy.dot is done for each step.
Pair list¶
-
class
tridesclous.gui.pairlist.PairList(controller=None, parent=None)[source]¶ Pair list is an intuituitive list of pair of cluster : when you click on a pair this make visible only this 2 cluster on all others views.
This help to validate in a fast way clusters.
- For convinience this paris can be filtered:
- all pairs : all combinaison
- high similarity: select only pairs that have similarity over a threshold (in settings)
- similarity amplitude ratio! select only pairs that have “similarity ratio” over a threshold (in settings)
- And they can be sorted by:
- label
- similarity
- ratio_similarity
- WIth a right click you can:
- merge a pair
- tag same cell, this keep 2 cluster but there tag as same cell.
Waveform viewer¶
-
class
tridesclous.gui.waveformviewer.WaveformViewer(controller=None, parent=None)[source]¶ Waveform viewer is undoubtedly the view to inspect waveforms.
Note that in some aspect Waveform hist viewer can be a better firend.
All centroid (median or mean) of visible cluster are plotted here.
- 2 main modes:
- geometry waveforms are organized with 2d geometry given by PRB file.
- flatten each chunk of each channel is put side by side in channel order than it can be ploted in 1d. The bottom view is th mad. On good cluster the mad must as close as possible from the value 1 because 1 is the normalized noise.
The geometry mode is more intuitive and help users about spatial information. But the flatten mode is really important because is give information about the variance (mad or std) for each point and about peak alignement.
The centoid is dfine by median+mad but you can also check with mean+std. For healthy cluster it should more or less the same.
- Important for zooming:
- geometry : zoomXY geometry = right click, move = left click and mouse wheel = zoom waveforms
- flatten: zoomXY = right click and move = left click
- Settings:
- plot_selected_spike: superimposed one slected peak on centroid
- show_only_selected_cluster: this auto hide all cluster except the one of selected spike
- plot_limit_for_flatten: for flatten mode this plot line for delimiting channels. Plotting is important but it slow down the zoom.
- metrics: choose median+mad or mean+std.
- show_channel_num*: what could it be ?
- flip_bottom_up: in geometry this flip bottom up the channel geometry.
- display_threshold: what could it be ?
Waveform hist viewer¶
-
class
tridesclous.gui.waveformhistviewer.WaveformHistViewer(controller=None, parent=None)[source]¶ Waveform histogram viewer is also a important thing.
It is equivalent to Waveform veiwer in flatten mode but with a 2d histogram that show the density (probability) of a cluster. So waveforms are flatten from (nb_peak, nb_sample, nb_channel) to (nb_peak, nb_channel*nb_sample) and are binarized on a 2d histogram. Then this is plotted as a map. The color code the density.
This is the best friend to see if two cluster are well discrimitated somewhere or if one cluster must be split.
- Important:
- use right click for X/Y zoom
- use left clik to move
- use mouse wheel for color zoom.Really important to play with this to discover low density
- intentionnaly not all cluster are displayed other we see nothing. The best is to plot 2 by 2. Furthermore it faster to plot with few cluster.
- don’t forget to display the noise snippet to validate that the mad is 1 for all channel.
- Settings:
- colormap hot is good because loaw density are black like background.
- data choose waveforms or features
- bin_min y limts of histogram
- bin_max y limts of histogram
- bin_size
- display_threshold
- max_label maximum number of labels displayed simulteneously (2 by default but you can set more)
Feature on time viewer¶
-
class
tridesclous.gui.waveformhistviewer.WaveformHistViewer(controller=None, parent=None)[source] Waveform histogram viewer is also a important thing.
It is equivalent to Waveform veiwer in flatten mode but with a 2d histogram that show the density (probability) of a cluster. So waveforms are flatten from (nb_peak, nb_sample, nb_channel) to (nb_peak, nb_channel*nb_sample) and are binarized on a 2d histogram. Then this is plotted as a map. The color code the density.
This is the best friend to see if two cluster are well discrimitated somewhere or if one cluster must be split.
- Important:
- use right click for X/Y zoom
- use left clik to move
- use mouse wheel for color zoom.Really important to play with this to discover low density
- intentionnaly not all cluster are displayed other we see nothing. The best is to plot 2 by 2. Furthermore it faster to plot with few cluster.
- don’t forget to display the noise snippet to validate that the mad is 1 for all channel.
- Settings:
- colormap hot is good because loaw density are black like background.
- data choose waveforms or features
- bin_min y limts of histogram
- bin_max y limts of histogram
- bin_size
- display_threshold
- max_label maximum number of labels displayed simulteneously (2 by default but you can set more)