U gfS@s(dZddlZddlZddlZddlZddlZddlZddlmZddl m Z ddl m Z ddl Z ddlmZddlmZddlmmZddlmZddlmZmZmZmZmZee dd Z!ee d d Z"ee d d Z#z ddl$Z$Wne%k rdZ$YnXej&d ddej&d e'dee j(dfddZ)dVddZ*ddZ+dWddZ,eedddZ-dd Z.dXd#d$Z/d%d&Z0dYd(d)Z1d*d+Z2d,d-Z3d.d/Z4ej5fd0d1Z6d2d3Z7dZd5d6Z8d7d8Z9d[d;d<Z:d\d?d@Z;d]dBdCZd`dPdQZ?GdRdSdSZ@GdTdUdUZAdS)azPyTorch utils.N)contextmanager)deepcopy)Path)DistributedDataParallel)LOGGER check_versioncolorstr file_date git_describe LOCAL_RANKRANK WORLD_SIZEignorezIUser provided device_type of 'cuda', but CUDA is not available. Disabling)message)categoryz1.9.0csfdd}|S)zbApplies torch.inference_mode() if torch>=1.9.0, else torch.no_grad() as a decorator for functions.csr tjntj|SN)torchinference_modeno_grad)fn torch_1_9_/home/ankuromar296_gmail_com/GitHubProjects/OPP/Palm_Detection/Code/yolov5/utils/torch_utils.pydecorate'sz&smart_inference_mode..decorater)rrrrrsmart_inference_mode$s rcCs:ttjdrtj|dS|dkr2td|dtS)zzReturns a CrossEntropyLoss with optional label smoothing for torch>=1.10.0; warns if smoothing on lower versions. z1.10.0label_smoothingruWARNING ⚠️ label smoothing z requires torch>=1.10.0)rr __version__nnCrossEntropyLossrwarningrrrrsmartCrossEntropyLoss-s   r%cCsJttjdddrtdttjdr6t|tgtddSt|tgtdSdS) zcInitializes DistributedDataParallel (DDP) for model training, respecting torch version constraints.1.12.0T)pinnedztorch==1.12.0 torchvision==0.13.0 DDP training is not supported due to a known issue. Please upgrade or downgrade torch to use DDP. See https://github.com/ultralytics/yolov5/issues/8395z1.11.0) device_ids output_device static_graph)r(r)N)rrr!AssertionErrorDDPr modelrrr smart_DDP8s  r/cCsnddlm}tt|dr|jn|d\}}t||r\|jj|krXt |jj ||_nt|t j r|j|krt ||t |j |nt|t j rjdd|D}t j |krt|d|dddt j }||j|krt ||j |||<nrt j|krjt|d|dddt j}||j|krjt j||j|||j||j||jdk d ||<dS) zeReshapes last layer of model to match class count 'n', supporting Classify, Linear, Sequential types.r)Classifyr.r cSsg|] }t|qSr)type.0xrrr Psz-reshape_classifier_output..rN)bias) models.commonr1listhasattrr.named_children isinstancelinear out_featuresr"Linear in_featuressetattr SequentiallenindexConv2d out_channels in_channels kernel_sizestrider7)r.nr1namemtypesirrrreshape_classifier_outputDs$ $     " "rO local_rankccs6|dkrtj|gddV|dkr2tjdgddS)zContext manager ensuring ordered operations in distributed training by making all processes wait for the leading process. )r r)r(Nr)distbarrierrPrrrtorch_distributed_zero_first[s rTcCsjtdkstdz:tdkr&dnd}ttj|ddddjdWSt k rdYd SXd S) zbReturns the number of available CUDA devices; works on Linux and Windows by invoking `nvidia-smi`.)LinuxWindowsz1device_count() only supported on Linux or WindowsrUznvidia-smi -L | wc -lznvidia-smi -L | find /c /v ""T)shellcapture_outputcheckr rN) platformsystemr+int subprocessrunstdoutdecodesplit Exception)cmdrrr device_countgs &rdTc Csdtp tdtdtjd}t| dd dd}|dk}|d k}|s^|rjd t j d <nB|r|t j d <tj rtj t| d dkstd |d|s|stj r|r|d nd}t|}|dkr|dkr||dkstd|d|dt|d}t|D]N\} } tj | } || dkrPdn|d| d| jd| jddd7}q,d} n<|rttddrtjj r|d7}d } n |d7}d} |s|}t|t| S)z_Selects computing device (CPU, CUDA GPU, MPS) for YOLOv5 model deployment, logging device info.u YOLOv5 🚀 z Python-z torch- zcuda:renonecpumpsz-1CUDA_VISIBLE_DEVICES,zInvalid CUDA '--device z<' requested, use '--device cpu' or pass valid CUDA device(s)0rrz batch-size z not multiple of GPU count zCUDA:z (, iz.0fzMiB) zcuda:0has_mpsFzMPS zCPU )r r rZpython_versionrr!strstriplowerreplaceosenvironcuda is_availablerdrCr+ra enumerateget_device_propertiesrK total_memorygetattrbackendsrirstriprinfodevice) r batch_sizenewlinesrhridevicesrJspacerNdpargrrr select_deviceqs@&     " <" rcCstjrtjtS)zeSynchronizes PyTorch for accurate timing, leveraging CUDA if available, and returns the current time.)rrvrw synchronizetimerrrr time_syncs  r c Csg}t|tjst|}tdddddddddddd d d t|trX|n|gD]}||}d |_t|tr|n|gD]T}t|d r||n|}t|d rt|tj r|j tj kr| n|}dddddg}}} z"t j||fddddd} Wntk r&d} YnXzrt|D]} t| d<||} t| d<z8t| trttdd| Dn| } t| d<Wn"tk rtd| d<YnX|| d| dd|7}|| d| dd|7}q4tjrtjdnd} dd|| fD\}}t|tjrDtdd|Dnd}t|d| d| d|d|dt|d t|d ||| | ||||gWn6tk r}zt||dW5d}~XYnXtjqq^|S)zYOLOv5 speed/memory/FLOPs profiler Usage: input = torch.randn(16, 3, 640, 640) m1 = lambda x: x * torch.sigmoid(x) m2 = nn.SiLU() profile(input, [m1, m2], n=100) # profile over 100 iterations Paramsz>12sGFLOPsz GPU_mem (GB)z>14sz forward (ms)z backward (ms)inputz>24soutputTtohalfrFinputsverboseeArcss|]}|VqdSr)sum)r4yirrr szprofile..nanr0css(|] }t|tjrt|jndVqdS)r9N)r<rTensortupleshaper3rrrrscss|]}|VqdSrnumelr3rrrrs12z12.4gz>14.3fz14.4gN)r<rrrprintr9r requires_gradr:rdtypefloat16rthopprofilerbrangerrbackwardfloatrvrwmemory_reservedr"Module parametersrpappend empty_cache)ropsrJrresultsr5rLtftbtflops_ymems_ins_outrerrrrsN , ."    * (:rcCst|tjjtjjfkS)zYChecks if the model is using Data Parallelism (DP) or Distributed Data Parallelism (DDP).)r2r"parallel DataParallelrr-rrr is_parallelsrcCst|r|jS|S)znReturns a single-GPU model by removing Data Parallelism (DP) or Distributed Data Parallelism (DDP) if applied.)rmoduler-rrr de_parallelsrcCs`|D]R}t|}|tjkr q|tjkr8d|_d|_q|tjtjtj tj tj fkrd|_ qdS)z|Initializes weights of Conv2d, BatchNorm2d, and activations (Hardswish, LeakyReLU, ReLU, ReLU6, SiLU) in the model. MbP?gQ?TN) modulesr2r"rE BatchNorm2depsmomentum Hardswish LeakyReLUReLUReLU6SiLUinplace)r.rLrrrrinitialize_weightss   rcsfddt|jDS)z_Finds and returns list of layer indices in `model.module_list` matching the specified `mclass`.csg|]\}}t|r|qSr)r<)r4rNrLmclassrrr6s z find_modules..)rx module_list)r.rrrr find_modulessrcCs:d\}}|D] }||7}||dk7}q||S)zzCalculates and returns the global sparsity of a model as the ratio of zero-valued parameters to total parameters. )rrr)rrr)r.abrrrrsparsitys   r333333?cCsnddlmmm}|D]0\}}t|tjr |j|d|d||dq t dt |dddS)zVPrunes Conv2d layers in a model to a specified sparsity using L1 unstructured pruning.rNweight)rKamountzModel pruned to z.3gz global sparsity) Ztorch.nn.utils.pruner"utilsprune named_modulesr<rEZl1_unstructuredremoverr~r)r.rrrKrLrrrrs  rc Cstj|j|j|j|j|j|j|jdd d |j j }|j |jd}t|j t|j|j}|j t|||j j|jdkrtj|j d|j j dn|j}|j|j |jt|j|j}|jt||ddd||S) z Fuses Conv2d and BatchNorm2d layers into a single Conv2d layer. See https://tehnokv.com/posts/fusing-batchnorm-and-conv/. T)rHrIpaddingdilationgroupsr7Fr Nrrr)r"rErGrFrHrIrrrrequires_grad_rrrcloneviewrdiagdivsqrtr running_varcopy_mmrr7zerossizemul running_meanreshape)convbn fusedconvw_convw_bnb_convb_bnrrrfuse_conv_and_bns,   *(&rFc Cstdd|D}tdd|D}|rtddddddd d dd d dd ddddddd t|D]F\}\}}|dd}td|||j|t|j | | fqzzt |}t |drtt|jdnd}tjd|j d||f|jd} tjt|| fddddd} t|trB|n||g}d| |d||d|dd } Wntk rd} YnXt |d!rt|jjd"d#nd$}t|d%tt|d&|d'|d(| d)S)*z Prints model summary including layers, parameters, gradients, and FLOPs; imgsz may be int or list. Example: img_size=640 or img_size=[640, 320] css|]}|VqdSrrr3rrrr.szmodel_info..css|]}|jr|VqdSr)rrr3rrrr/slayerz>5rfrKz>40gradientz>9rz>12rz>20muz>10sigmaz module_list.rez$%5g %40s %9s %12g %20s %10.3g %10.3grI rrFrrrrrmz.1fz GFLOPs yaml_fileyolov5ZYOLOv5Modelz summary: z layers, z parameters, z gradientsN) rrrrxnamed_parametersrsrrr9rmeanstdnextr:maxr\rIremptyrrrrr<rbrrstemrr~rCr) r.rimgszn_pn_grNrKrrIimrfsrrr model_info(s,> $ ""* $r ?rcsdkr |S|jdd\}}t|t|f}tj||ddd}|shfdd||fD\}}tj|d ||d d ||d gd d S) zScales an image tensor `img` of shape (bs,3,y,x) by `ratio`, optionally maintaining the original shape, padded to multiples of `gs`. r rNbilinearF)rmode align_cornersc3s$|]}t|VqdSr)mathceilr3gsratiorrrQszscale_img..rrgS㥛?)value)rr\F interpolatepad)imgr same_shaperhwrrrr scale_imgGsrrcCsJ|jD]:\}}t|r"||ks |ds ||kr8q q t|||q dS)zZCopies attributes from object b to a, optionally filtering with include and exclude lists.rN)__dict__itemsrC startswithrA)rrincludeexcludekvrrr copy_attrUs"r$Adamr?h㈵>c Csgggf}tddtjD}|D]b}|jddD]P\}} |dkrZ|d| q:|dkr|t||r||d| q:|d| q:q*|d krtj j |d||d fd } nz|d krtj j |d||d fd d} nT|dkrtj j |d||d} n4|dkrtj j |d||dd} ntd|d| |d|d| |dd dttddt| jd|dt|ddt|dd|dt|dd| S)z Initializes YOLOv5 smart optimizer with 3 parameter groups for different decay configurations. Groups are 0) weights with decay, 1) weights no decay, 2) biases no decay. css|]\}}d|kr|VqdS)NormNr)r4r"r#rrrresz"smart_optimizer..r)recurser7rrrr%g+?)lrbetasAdamWr)r*r+ weight_decayRMSProp)r*rSGDT)r*rnesterovz Optimizer z not implemented.)paramsr-z optimizer:rfz(lr=z) with parameter groups z weight(decay=0.0), z weight(decay=z), z bias)rr"rrrrrr<roptimr%r,RMSpropr/NotImplementedErroradd_param_grouprr~rr2__name__rC) r.rKr*rdecaygrr#p_namer optimizerrrrsmart_optimizer^s0   Nr;ultralytics/yolov5yolov5scKspttjdrd|d<ttjdr(d|d<ztjj||f|WStk rjtjj||fddi|YSXdS)zgYOLOv5 torch.hub.load() wrapper with smart error handling, adjusting torch arguments for compatibility.z1.9.1Tskip_validationr& trust_repo force_reloadN)rrr!hubloadrb)repor.kwargsrrrsmart_hub_loads  rE yolov5s.pt,c Csd}|dd}|ddk r2||d|d}|rb|drb|j|d|d|_|r|d kst|d |d |d td |d|d|d||krt|d|dd|d||d7}|||fS)zkResumes training from a checkpoint, updating optimizer, ema, and epochs, with optional resume verification.repochrr:N best_fitnessemaupdatesrz training to zo epochs is finished, nothing to resume. Start a new training without --resume, i.e. 'python train.py --weights 'zResuming training from z from epoch z to z total epochsz has been trained for z epochs. Fine-tuning for z more epochs.) load_state_dictgetrJr state_dictrKr+rr~)ckptr:rJweightsepochsresumerI start_epochrrr smart_resumes"      rUc@seZdZdddZddZdS) EarlyStoppingcCs$d|_d|_|ptd|_d|_dS)zjInitializes simple early stopping mechanism for YOLOv5, with adjustable patience for non-improving epochs.rrinfFN)rI best_epochrpatience possible_stop)selfrZrrr__init__szEarlyStopping.__init__c Csf||jkr||_||_||j}||jdk|_||jk}|rbtd|jd|jd|jd|S)zaEvaluates if training should stop based on fitness improvement and patience, returning a boolean.rz;Stopping training early as no improvement observed in last z( epochs. Best results observed at epoch z@, best model saved as best.pt. To update EarlyStopping(patience=zr) pass a new patience value, i.e. `python train.py --patience 300` or use `--patience 0` to disable EarlyStopping.)rIrYrZr[rr~)r\rHfitnessdeltastoprrr__call__s   zEarlyStopping.__call__N)rW)r6 __module__ __qualname__r]rarrrrrVs rVc@s,eZdZdZdddZddZdd d Zd S)ModelEMAa#Updated Exponential Moving Average (EMA) from https://github.com/rwightman/pytorch-image-models Keeps a moving average of everything in the model state_dict (parameters and buffers) For EMA details see https://www.tensorflow.org/api_docs/python/tf/train/ExponentialMovingAverage H.?rcsFtt||_||_fdd|_|jD]}|dq2dS)zInitializes EMA with model parameters, decay rate, tau for decay adjustment, and update count; sets model to evaluation mode. csdt| S)Nr)rexp)r5r7taurrz#ModelEMA.__init__..FN)rrevalrJrKr7rr)r\r.r7rirKrrrhrr]s zModelEMA.__init__cCsj|jd7_||j}t|}|jD]0\}}|jjr4||9}|d|||7}q4dS)z`Updates the Exponential Moving Average (EMA) parameters based on the current model's parameters.rN) rKr7rrOrJrris_floating_pointdetach)r\r.rmsdr"r#rrrupdates  zModelEMA.updater process_groupreducercCst|j|||dS)zUpdates EMA attributes by copying specified attributes from model to EMA, excluding certain attributes by default. N)r$rJ)r\r.r r!rrr update_attrszModelEMA.update_attrN)rerfr)rrq)r6rbrc__doc__r]rprtrrrrrds  rd)r)r0)rerT)rN)r)Fr)r Fr)rr)r%rr&r')r<r=)NrFrGT)BrurrtrZr]rwarnings contextlibrcopyrpathlibrrtorch.distributed distributedrRtorch.nnr"Ztorch.nn.functional functionalrZtorch.nn.parallelrr, utils.generalrrrr r r\getenvr r rr ImportErrorfilterwarnings UserWarningr!rr%r/rOrTrdrrrrrrrErrrrr rr$r;rErUrVrdrrrrsb            % 5 "   %