a ߙfb @sddlZddlZddlmZddlmZddlmZddl m Z ddl m Z ddl mZddlmZerpddlZgd Zd d Zd!d d Zd"ddZd#ddZd$ddZGdddZd%ddZd&dd ZdS)'N)normalize_axis_index)Quantity) isiterable)AstropyUserWarning)_sigma_clip_fastmad_std)HAS_BOTTLENECK) SigmaClip sigma_clipsigma_clipped_statscs`t}tfddt|jD7tt|j}t||}|d|j|d}|S)z Bottleneck can only take integer axis, not tuple, so this function takes all the axes to be operated on and combines them into the first dimension of the array so that we can then use axis=0. c3s|]}|vr|VqdSN).0iaxisr;lib/python3.9/site-packages/astropy/stats/sigma_clipping.py z)_move_tuple_axes_first..N)lentuplerangendimnpZmoveaxisreshapeshape)arrayrZnaxisZ destinationZ array_newrrr_move_tuple_axes_firsts  r cCsJt|trt||d}d}t|tr8|tj||dStj||dSdS)z3Bottleneck nanmean function that handle tuple axis.rrN) isinstancerr r__array_wrap__ bottlenecknanmeanrrrrr_nanmean0s    r&cCsJt|trt||d}d}t|tr8|tj||dStj||dSdS)z5Bottleneck nanmedian function that handle tuple axis.rrN)r!rr rr"r# nanmedianr%rrr _nanmedian=s    r(cCsNt|trt||d}d}t|tr:|tj|||dStj|||dSdS)z2Bottleneck nanstd function that handle tuple axis.rr)rddofN)r!rr rr"r#nanstd)rrr)rrr_nanstdJs    r+cCst||ddS)z.mad_std function that ignores NaNs by default.T)rZ ignore_nanrr%rrr _nanmadstdXsr,c@steZdZdZddd Zd d Zd d ZeddZeddZ dddZ dddZ d ddZ d!ddZ d"ddZdS)#r a< Class to perform sigma clipping. The data will be iterated over, each time rejecting values that are less or more than a specified number of standard deviations from a center value. Clipped (rejected) pixels are those where:: data < center - (sigma_lower * std) data > center + (sigma_upper * std) where:: center = cenfunc(data [, axis=]) std = stdfunc(data [, axis=]) Invalid data values (i.e., NaN or inf) are automatically clipped. For a functional interface to sigma clipping, see :func:`sigma_clip`. .. note:: `scipy.stats.sigmaclip` provides a subset of the functionality in this class. Also, its input data cannot be a masked array and it does not handle data that contains invalid values (i.e., NaN or inf). Also note that it uses the mean as the centering function. The equivalent settings to `scipy.stats.sigmaclip` are:: sigclip = SigmaClip(sigma=4., cenfunc='mean', maxiters=None) sigclip(data, axis=None, masked=False, return_bounds=True) Parameters ---------- sigma : float, optional The number of standard deviations to use for both the lower and upper clipping limit. These limits are overridden by ``sigma_lower`` and ``sigma_upper``, if input. The default is 3. sigma_lower : float or None, optional The number of standard deviations to use as the lower bound for the clipping limit. If `None` then the value of ``sigma`` is used. The default is `None`. sigma_upper : float or None, optional The number of standard deviations to use as the upper bound for the clipping limit. If `None` then the value of ``sigma`` is used. The default is `None`. maxiters : int or None, optional The maximum number of sigma-clipping iterations to perform or `None` to clip until convergence is achieved (i.e., iterate until the last iteration clips nothing). If convergence is achieved prior to ``maxiters`` iterations, the clipping iterations will stop. The default is 5. cenfunc : {'median', 'mean'} or callable, optional The statistic or callable function/object used to compute the center value for the clipping. If using a callable function/object and the ``axis`` keyword is used, then it must be able to ignore NaNs (e.g., `numpy.nanmean`) and it must have an ``axis`` keyword to return an array with axis dimension(s) removed. The default is ``'median'``. stdfunc : {'std', 'mad_std'} or callable, optional The statistic or callable function/object used to compute the standard deviation about the center value. If using a callable function/object and the ``axis`` keyword is used, then it must be able to ignore NaNs (e.g., `numpy.nanstd`) and it must have an ``axis`` keyword to return an array with axis dimension(s) removed. The default is ``'std'``. grow : float or `False`, optional Radius within which to mask the neighbouring pixels of those that fall outwith the clipping limits (only applied along ``axis``, if specified). As an example, for a 2D image a value of 1 will mask the nearest pixels in a cross pattern around each deviant pixel, while 1.5 will also reject the nearest diagonal neighbours and so on. See Also -------- sigma_clip, sigma_clipped_stats Notes ----- The best performance will typically be obtained by setting ``cenfunc`` and ``stdfunc`` to one of the built-in functions specified as as string. If one of the options is set to a string while the other has a custom callable, you may in some cases see better performance if you have the `bottleneck`_ package installed. .. _bottleneck: https://github.com/pydata/bottleneck Examples -------- This example uses a data array of random variates from a Gaussian distribution. We clip all points that are more than 2 sample standard deviations from the median. The result is a masked array, where the mask is `True` for clipped data:: >>> from astropy.stats import SigmaClip >>> from numpy.random import randn >>> randvar = randn(10000) >>> sigclip = SigmaClip(sigma=2, maxiters=5) >>> filtered_data = sigclip(randvar) This example clips all points that are more than 3 sigma relative to the sample *mean*, clips until convergence, returns an unmasked `~numpy.ndarray`, and modifies the data in-place:: >>> from astropy.stats import SigmaClip >>> from numpy.random import randn >>> from numpy import mean >>> randvar = randn(10000) >>> sigclip = SigmaClip(sigma=3, maxiters=None, cenfunc='mean') >>> filtered_data = sigclip(randvar, masked=False, copy=False) This example sigma clips along one axis:: >>> from astropy.stats import SigmaClip >>> from numpy.random import normal >>> from numpy import arange, diag, ones >>> data = arange(5) + normal(0., 0.05, (5, 5)) + diag(ones(5)) >>> sigclip = SigmaClip(sigma=2.3) >>> filtered_data = sigclip(data, axis=0) Note that along the other axis, no points would be clipped, as the standard deviation is higher. @NmedianstdFc Cs||_|p ||_|p||_|p"tj|_||_||_|||_ | ||_ tj |_ tj |_d|_||_|jr~ddlm}||_dS)Nr)binary_dilation)sigma sigma_lower sigma_upperrinfmaxiterscenfuncstdfunc_parse_cenfunc_cenfunc_parsed_parse_stdfunc_stdfunc_parsednan _min_value _max_value _niterationsgrowZ scipy.ndimager1_binary_dilation) selfr2r3r4r6r7r8rAr1rrr__init__s      zSigmaClip.__init__c Cs,d|j|j|j|jt|jt|j|jS)NzaSigmaClip(sigma={}, sigma_lower={}, sigma_upper={}, maxiters={}, cenfunc={}, stdfunc={}, grow={})) formatr2r3r4r6reprr7r8rA)rCrrr__repr__s  zSigmaClip.__repr__c CsNd|jjdg}gd}|D]$}|d|dtt||qd|S)N<>r2r3r4r6r7r8rAz z:  ) __class____name__appendrFgetattrjoin)rClinesattrsattrrrr__str__s "zSigmaClip.__str__cCsPt|trL|dkr$trt}qLtj}n(|dkr>tr6t}qLtj}nt|d|S)Nr/meanz is an invalid cenfunc.) r!strr r(rr'r&r$ ValueError)r7rrrr9s zSigmaClip._parse_cenfunccCsDt|tr@|dkr$trt}q@tj}n|dkr2t}nt|d|S)Nr0rz is an invalid stdfunc.)r!rVr r+rr*r,rW)r8rrrr;s zSigmaClip._parse_stdfunccCs~tbtjdtd|j||d|_|j||d}|j||j|_|j||j 7_Wdn1sp0YdS)Nignore)categoryr) warningscatch_warnings simplefilterRuntimeWarningr:r?r<r3r>r4)rCdatarr0rrr_compute_bounds)s  zSigmaClip._compute_boundsTc sttrjj}nd}|durB|durBjjdkrBtddurfjdkrXdn tt jt st j}d}njtfddDtfd dt jD } | j }| |djtd }d|jjdks |jjd kr|t}t|} t| r8td tt|tjjr|| |jO} tj|tj}t| |j } t || |j!d k|j"dkt#|j$rdn|j$|j%|j&d\} } tj'dd8| |t(| kO} | |t(| kO} Wdn1s0Y|durH| |} | tfddt jD} |rbtjj)| |d} n$|rxjtdd} n} tj*| | <|dur| |>} | |>} | |>} |r| | | fS| SdS)z= Fast C implementation for simple use cases. NFfzacannot mask non-floating-point array with NaN values, set copy=True or masked=True to avoid this.rc3s|]}t|jVqdSr )rrrZax)r^rrrNrz,SigmaClip._sigmaclip_fast..c3s|]}|vr|VqdSr rrbrrrrOsrTInput data contains invalid values (NaNs or infs), which were automatically clipped.r/rrrXZinvalidc3s|]}|VqdSr )indexrb)transposed_axesrrrss)maskcopyTri)+r!rvalueunitZdtypekind ExceptionrrrrrZ transposerrritemsizeastypefloatrisfiniteanyrZwarnrma MaskedArrayrhZviewZndarrayZ broadcast_torirr7r8Zisinfr6r3r4errstateZ expand_dimsrr=)rCr^rmasked return_boundsrirlZ data_reshapedZtransposed_shapeZdata_transposedrhZbound_loZbound_hiresultr)rr^rgr_sigmaclip_fast3sx           4      zSigmaClip._sigmaclip_fastc Cs0|}t|tjjr$|j|j}t|}t|rN||}t dt d}d}|dkr||j kr|d7}|j } |j|dd|||jk||jk@}| |j }qV||_|rtjj||d}tjdd2|jt||jk||jkO_Wdn1s 0Y|r(||j|jfS|SdS) z Sigma clip when ``axis`` is None and ``grow`` is not >0. In this simple case, we remove clipped elements from the flattened array during each iteration. rdrarNrrjrXre)Zravelr!rrurvr^rhrrrsrZrtrr6sizer_r>r?r@masked_invalidrw logical_or) rCr^rxryri filtered_dataZ good_masknchanged iterationr|rrr_sigmaclip_noaxiss>   (zSigmaClip._sigmaclip_noaxiscs|tt}t|r6tj|<tdtt tj j rbtj t tjdurtsxftfddDtfddtjD}|jr2t|jdd}tjtd|f|j} durt| D]\} } | vr|| | k<qtfd d| D|jdk} ~ d} d}| dkr||jkr|d7}|jd t|js|j||_|j||_tjd d $|jk|jkB}Wdn1s0Y|jr||| }tj|<t |} ~q:||_!|r|r.tj j |td dndtjd d Ftj j |dd}tj j"t#||jk||jk|ddWdn1s0Y|r|j|jfSSdS)z Sigma clip the data when ``axis`` or ``grow`` is specified. In this case, we replace clipped values with NaNs as placeholder values. rdNc3s$|]}|dkrj|n|VqdS)rN)r)rn)rrrrrz0SigmaClip._sigmaclip_withaxis..c3s"|]\}}|vrdn|VqdS)raNr)rdimr|rrrrsrarc3s|]}|dVqdS)rNr)ridx)cenidxrrrrrrXreTrjF)$rprqrrrrsr=rZrtrr!rurvr}Zfilledrr enumeraterrAintZmgridslicersumr6r_Zisscalarr>rr?rwrBZ count_nonzeror@Z masked_wherer~)rCr^rrxryriZbad_maskZmshaper|indicesrrZkernelrrZnew_maskoutr)rrrr_sigmaclip_withaxiss~         $     &zSigmaClip._sigmaclip_withaxiscCst|}|jdkr@|r&tj|}n|}|r<||j|jfS|St|tjjr|j r|rb|}nt |j tj }|r||j|jfS|S|j dvr|jdvr|dur|js|j|||||dS|dur|js|j||||dS|j|||||dSdS)a Perform sigma clipping on the provided data. Parameters ---------- data : array-like or `~numpy.ma.MaskedArray` The data to be sigma clipped. axis : None or int or tuple of int, optional The axis or axes along which to sigma clip the data. If `None`, then the flattened data will be used. ``axis`` is passed to the ``cenfunc`` and ``stdfunc``. The default is `None`. masked : bool, optional If `True`, then a `~numpy.ma.MaskedArray` is returned, where the mask is `True` for clipped values. If `False`, then a `~numpy.ndarray` is returned. The default is `True`. return_bounds : bool, optional If `True`, then the minimum and maximum clipping bounds are also returned. copy : bool, optional If `True`, then the ``data`` array will be copied. If `False` and ``masked=True``, then the returned masked array data will contain the same array as the input ``data`` (if ``data`` is a `~numpy.ndarray` or `~numpy.ma.MaskedArray`). If `False` and ``masked=False``, the input data is modified in-place. The default is `True`. Returns ------- result : array-like If ``masked=True``, then a `~numpy.ma.MaskedArray` is returned, where the mask is `True` for clipped values and where the input mask was `True`. If ``masked=False``, then a `~numpy.ndarray` is returned. If ``return_bounds=True``, then in addition to the masked array or array above, the minimum and maximum clipping bounds are returned. If ``masked=False`` and ``axis=None``, then the output array is a flattened 1D `~numpy.ndarray` where the clipped values have been removed. If ``return_bounds=True`` then the returned minimum and maximum thresholds are scalars. If ``masked=False`` and ``axis`` is specified, then the output `~numpy.ndarray` will have the same shape as the input ``data`` and contain ``np.nan`` where values were clipped. If the input ``data`` was a masked array, then the output `~numpy.ndarray` will also contain ``np.nan`` where the input mask was `True`. If ``return_bounds=True`` then the returned minimum and maximum clipping thresholds will be be `~numpy.ndarray`\s. r)rUr/)r0rNrrxryri)rxryri)rZ asanyarrayr|rurvr>r?r!rhallZfullrr=r7r8rAr{rr)rCr^rrxryrirzrrr__call__sF<     zSigmaClip.__call__)r-NNr.r/r0F)N)NTFT)TFT)NTFT)NTFT)rM __module__ __qualname____doc__rDrGrT staticmethodr9r;r_r{rrrrrrrr ]s2     V 2 ar r.r/r0TFc Cs(t||||||| d} | |||| | dS)a5 Perform sigma-clipping on the provided data. The data will be iterated over, each time rejecting values that are less or more than a specified number of standard deviations from a center value. Clipped (rejected) pixels are those where:: data < center - (sigma_lower * std) data > center + (sigma_upper * std) where:: center = cenfunc(data [, axis=]) std = stdfunc(data [, axis=]) Invalid data values (i.e., NaN or inf) are automatically clipped. For an object-oriented interface to sigma clipping, see :class:`SigmaClip`. .. note:: `scipy.stats.sigmaclip` provides a subset of the functionality in this class. Also, its input data cannot be a masked array and it does not handle data that contains invalid values (i.e., NaN or inf). Also note that it uses the mean as the centering function. The equivalent settings to `scipy.stats.sigmaclip` are:: sigma_clip(sigma=4., cenfunc='mean', maxiters=None, axis=None, ... masked=False, return_bounds=True) Parameters ---------- data : array-like or `~numpy.ma.MaskedArray` The data to be sigma clipped. sigma : float, optional The number of standard deviations to use for both the lower and upper clipping limit. These limits are overridden by ``sigma_lower`` and ``sigma_upper``, if input. The default is 3. sigma_lower : float or None, optional The number of standard deviations to use as the lower bound for the clipping limit. If `None` then the value of ``sigma`` is used. The default is `None`. sigma_upper : float or None, optional The number of standard deviations to use as the upper bound for the clipping limit. If `None` then the value of ``sigma`` is used. The default is `None`. maxiters : int or None, optional The maximum number of sigma-clipping iterations to perform or `None` to clip until convergence is achieved (i.e., iterate until the last iteration clips nothing). If convergence is achieved prior to ``maxiters`` iterations, the clipping iterations will stop. The default is 5. cenfunc : {'median', 'mean'} or callable, optional The statistic or callable function/object used to compute the center value for the clipping. If using a callable function/object and the ``axis`` keyword is used, then it must be able to ignore NaNs (e.g., `numpy.nanmean`) and it must have an ``axis`` keyword to return an array with axis dimension(s) removed. The default is ``'median'``. stdfunc : {'std', 'mad_std'} or callable, optional The statistic or callable function/object used to compute the standard deviation about the center value. If using a callable function/object and the ``axis`` keyword is used, then it must be able to ignore NaNs (e.g., `numpy.nanstd`) and it must have an ``axis`` keyword to return an array with axis dimension(s) removed. The default is ``'std'``. axis : None or int or tuple of int, optional The axis or axes along which to sigma clip the data. If `None`, then the flattened data will be used. ``axis`` is passed to the ``cenfunc`` and ``stdfunc``. The default is `None`. masked : bool, optional If `True`, then a `~numpy.ma.MaskedArray` is returned, where the mask is `True` for clipped values. If `False`, then a `~numpy.ndarray` and the minimum and maximum clipping thresholds are returned. The default is `True`. return_bounds : bool, optional If `True`, then the minimum and maximum clipping bounds are also returned. copy : bool, optional If `True`, then the ``data`` array will be copied. If `False` and ``masked=True``, then the returned masked array data will contain the same array as the input ``data`` (if ``data`` is a `~numpy.ndarray` or `~numpy.ma.MaskedArray`). If `False` and ``masked=False``, the input data is modified in-place. The default is `True`. grow : float or `False`, optional Radius within which to mask the neighbouring pixels of those that fall outwith the clipping limits (only applied along ``axis``, if specified). As an example, for a 2D image a value of 1 will mask the nearest pixels in a cross pattern around each deviant pixel, while 1.5 will also reject the nearest diagonal neighbours and so on. Returns ------- result : array-like If ``masked=True``, then a `~numpy.ma.MaskedArray` is returned, where the mask is `True` for clipped values and where the input mask was `True`. If ``masked=False``, then a `~numpy.ndarray` is returned. If ``return_bounds=True``, then in addition to the masked array or array above, the minimum and maximum clipping bounds are returned. If ``masked=False`` and ``axis=None``, then the output array is a flattened 1D `~numpy.ndarray` where the clipped values have been removed. If ``return_bounds=True`` then the returned minimum and maximum thresholds are scalars. If ``masked=False`` and ``axis`` is specified, then the output `~numpy.ndarray` will have the same shape as the input ``data`` and contain ``np.nan`` where values were clipped. If the input ``data`` was a masked array, then the output `~numpy.ndarray` will also contain ``np.nan`` where the input mask was `True`. If ``return_bounds=True`` then the returned minimum and maximum clipping thresholds will be be `~numpy.ndarray`\s. See Also -------- SigmaClip, sigma_clipped_stats Notes ----- The best performance will typically be obtained by setting ``cenfunc`` and ``stdfunc`` to one of the built-in functions specified as as string. If one of the options is set to a string while the other has a custom callable, you may in some cases see better performance if you have the `bottleneck`_ package installed. .. _bottleneck: https://github.com/pydata/bottleneck Examples -------- This example uses a data array of random variates from a Gaussian distribution. We clip all points that are more than 2 sample standard deviations from the median. The result is a masked array, where the mask is `True` for clipped data:: >>> from astropy.stats import sigma_clip >>> from numpy.random import randn >>> randvar = randn(10000) >>> filtered_data = sigma_clip(randvar, sigma=2, maxiters=5) This example clips all points that are more than 3 sigma relative to the sample *mean*, clips until convergence, returns an unmasked `~numpy.ndarray`, and does not copy the data:: >>> from astropy.stats import sigma_clip >>> from numpy.random import randn >>> from numpy import mean >>> randvar = randn(10000) >>> filtered_data = sigma_clip(randvar, sigma=3, maxiters=None, ... cenfunc=mean, masked=False, copy=False) This example sigma clips along one axis:: >>> from astropy.stats import sigma_clip >>> from numpy.random import normal >>> from numpy import arange, diag, ones >>> data = arange(5) + normal(0., 0.05, (5, 5)) + diag(ones(5)) >>> filtered_data = sigma_clip(data, sigma=2.3, axis=0) Note that along the other axis, no points would be clipped, as the standard deviation is higher. rJr)r ) r^r2r3r4r6r7r8rrxryrirAsigcliprrrr s9r r-c  Cs|durtj||}|dur,tj||}t|tjjrZ|jrZtjjtjjtjjfSt||||||| d} | || dddd} t rt | | d}t | | d}t | | | d}n,tj | | d}tj| | d}tj| | | d}|||fS)a# Calculate sigma-clipped statistics on the provided data. Parameters ---------- data : array-like or `~numpy.ma.MaskedArray` Data array or object that can be converted to an array. mask : `numpy.ndarray` (bool), optional A boolean mask with the same shape as ``data``, where a `True` value indicates the corresponding element of ``data`` is masked. Masked pixels are excluded when computing the statistics. mask_value : float, optional A data value (e.g., ``0.0``) that is ignored when computing the statistics. ``mask_value`` will be masked in addition to any input ``mask``. sigma : float, optional The number of standard deviations to use for both the lower and upper clipping limit. These limits are overridden by ``sigma_lower`` and ``sigma_upper``, if input. The default is 3. sigma_lower : float or None, optional The number of standard deviations to use as the lower bound for the clipping limit. If `None` then the value of ``sigma`` is used. The default is `None`. sigma_upper : float or None, optional The number of standard deviations to use as the upper bound for the clipping limit. If `None` then the value of ``sigma`` is used. The default is `None`. maxiters : int or None, optional The maximum number of sigma-clipping iterations to perform or `None` to clip until convergence is achieved (i.e., iterate until the last iteration clips nothing). If convergence is achieved prior to ``maxiters`` iterations, the clipping iterations will stop. The default is 5. cenfunc : {'median', 'mean'} or callable, optional The statistic or callable function/object used to compute the center value for the clipping. If using a callable function/object and the ``axis`` keyword is used, then it must be able to ignore NaNs (e.g., `numpy.nanmean`) and it must have an ``axis`` keyword to return an array with axis dimension(s) removed. The default is ``'median'``. stdfunc : {'std', 'mad_std'} or callable, optional The statistic or callable function/object used to compute the standard deviation about the center value. If using a callable function/object and the ``axis`` keyword is used, then it must be able to ignore NaNs (e.g., `numpy.nanstd`) and it must have an ``axis`` keyword to return an array with axis dimension(s) removed. The default is ``'std'``. std_ddof : int, optional The delta degrees of freedom for the standard deviation calculation. The divisor used in the calculation is ``N - std_ddof``, where ``N`` represents the number of elements. The default is 0. axis : None or int or tuple of int, optional The axis or axes along which to sigma clip the data. If `None`, then the flattened data will be used. ``axis`` is passed to the ``cenfunc`` and ``stdfunc``. The default is `None`. grow : float or `False`, optional Radius within which to mask the neighbouring pixels of those that fall outwith the clipping limits (only applied along ``axis``, if specified). As an example, for a 2D image a value of 1 will mask the nearest pixels in a cross pattern around each deviant pixel, while 1.5 will also reject the nearest diagonal neighbours and so on. Notes ----- The best performance will typically be obtained by setting ``cenfunc`` and ``stdfunc`` to one of the built-in functions specified as as string. If one of the options is set to a string while the other has a custom callable, you may in some cases see better performance if you have the `bottleneck`_ package installed. .. _bottleneck: https://github.com/pydata/bottleneck Returns ------- mean, median, stddev : float The mean, median, and standard deviation of the sigma-clipped data. See Also -------- SigmaClip, sigma_clip NrJFTrr)r)r)rrurvZ masked_valuesr!rhrrxr r r&r(r+r$r'r*)r^rhZ mask_valuer2r3r4r6r7r8Zstd_ddofrrArZ data_clippedrUr/r0rrrr Gs*c   r )N)N)Nr)N) rNNr.r/r0NTFTF) NNr-NNr.r/r0rNF)rZZnumpyrZnumpy.core.multiarrayrZ astropy.unitsrZ astropy.utilsrZastropy.utils.exceptionsrZastropy.stats._fast_sigma_cliprZastropy.stats.funcsrZ"astropy.utils.compat.optional_depsr r#__all__r r&r(r+r,r r r rrrrs>         . A