[i7 UdZddlZddlZddlmZddlmZmZmZddl Z ddl m Z m Z ddl mZmZmZddlmZdd lmZee eefZe jZe jZd Zee jjd fed <d aga e!eegdfe!efed<ej"Z#ej$Z%eZ&e'e jdr e jj(Z)ndedefdZ)e'e jdr e jj*Z+ndedefdZ+dZ,dZ-de.fdZ/d@dZ0Gdde1Z2de.fdZ3de.fdZ4dAdeeddfdZ5defdZ6defdZ7dAdeedefd Z8dAdeedefd!Z9 dBd$eed%ed&eddfd'Z:de;eeffd(ZdAdeedeeeffd.Z?d@d/Z@d0efd1ZAdeddfd2ZBdAdeede;eeffd3ZCGd4dZ Gd5d6ZDd7ZEd0ed8deDfd9ZFdDdeeee j fde fd;ZG dDdlITdd>lJTgd?ZKdS)EzH This package enables an interface for accessing MTIA backend in python N)Callable)AnyOptionalUnion)deviceTensor) _dummy_type_LazySeedTracker classproperty)Device)_get_device_index.default_generatorsF _queued_calls_mtia_exchangeDevicerreturnc0|dkrdStdNrz)PyTorch was compiled without MTIA support RuntimeErrorrs F/var/www/icac/venv/lib/python3.11/site-packages/torch/mtia/__init__.py_exchange_devicer( A::2FGGG_mtia_maybeExchangeDevicec0|dkrdStdrrrs r_maybe_exchange_devicer 2rrc"tdSN) _lazy_initrrrinitr$8sLLLLLrc.tot S)z9Return whether PyTorch's MTIA state has been initialized.) _initialized_is_in_bad_forkrrris_initializedr(<s  1 1 111rc>tjSr")torch_C_mtia_isInBadForkrrrr'r'As 8 % % ' ''rc 2tsttdrdSt5tr ddddSt rt dt stdtj dt_ t dtD tD]Z\}} |#t $r=}dt#|dd|}t'||d}~wwxYw t)tdn#t)tdwxYwdaddddS#1swxYwYdS) Nis_initializingzwCannot re-initialize MTIA in forked subprocess. To use MTIA with multiprocessing, you must use the 'spawn' start methodzTorch not compiled with MTIA enabled. Ensure you have `import mtia.host_runtime.torch_mtia.dynamic_library` in your python src file and include `//mtia/host_runtime/torch_mtia:torch_mtia` as your target dependency!Tc3K|]}||V dSr"r).0callss r z_lazy_init..gs(XXuRWXUXXXXXXrz6MTIA call failed lazily at initialization with error: z( MTIA call was originally invoked at: )r(hasattr_tls_initialization_lockr'r _is_compiledAssertionErrorr*r+ _mtia_initr.rextend_lazy_seed_tracker get_calls ExceptionstrjoinDeferredMtiaCallErrordelattrr&) queued_callorig_tracebackemsgs rr#r#Es;74):;; ,,     ,,,,,,,,    I ~~  *   $XX0B0L0L0N0NXXXXXX -/< < <+ ^<KMMMM <<<]QTUVQWQW]]CE77>CZCZ]]044!; < < D+ , , , ,GD+ , , , , Y,,,,,,,,,,,,,,,,,,sTF  BF * E#7 DE# E  8EE  E# F #E::F  FFceZdZdS)r@N)__name__ __module__ __qualname__rrrr@r@xsDrr@c>tjS)z*Return true if compiled with MTIA support.)r*r+ _mtia_isBuiltrrrr7r7|s 8 ! ! # ##rcFtsdStdkS)z'Return true if MTIA device is availableFr)r7 device_countrrr is_availablerNs" >>u >>A rctj|5tjcdddS#1swxYwYdS)z?Waits for all jobs in all streams on a MTIA device to complete.N)r*mtiarr+_mtia_deviceSynchronizers r synchronizerRs   6 " "22x//11222222222222222222sA  AAc>tjS)z,Return the number of MTIA devices available.)r*r+_mtia_getDeviceCountrrrrMrM 8 ( ( * **rc>tjS)z0Return the index of a currently selected device.)r*r+%_accelerator_hooks_get_current_devicerrrcurrent_devicerXs 8 9 9 ; ;;rc^tjt|dS)aSReturn the currently selected :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the currently selected :class:`Stream` for the current device, given by :func:`~torch.mtia.current_device`, if :attr:`device` is ``None`` (default). Toptional)r*r+_mtia_getCurrentStreamrrs rcurrent_streamr]( 8 * *+tjS)z4Return a dictionary of MTIA memory allocator history)r*r+_mtia_memorySnapshotrrrsnapshotrlrUrobservercDtj|dS)z9Attach an out-of-memory observer to MTIA memory allocatorN)r*r+_mtia_attachOutOfMemoryObserver)rms rattach_out_of_memory_observerrps  H,,X66666rTincluding_emulationcdS)zLReturn a bool indicating if the current MTIA device supports dtype bfloat16.Tr)rqs ris_bf16_supportedrss 4rc^tjt|dS)aReturn capability of a given device as a tuple of (major version, minor version). Args: device (torch.device or int, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). TrZ)r*r+_mtia_getDeviceCapabilityrrs rget_device_capabilityrv) 8 - -.?QU.V.V.V W WWrc>tjS)zEmpty the MTIA device cache.)r*r+_mtia_emptyCacherrr empty_cacherzs 8 $ $ & &&rstreamcL|dStj|dS)aSet the current stream.This is a wrapper API to set the stream. Usage of this function is discouraged in favor of the ``stream`` context manager. Args: stream (Stream): selected stream. This function is a no-op if this argument is ``None``. N)r*r+_mtia_setCurrentStreamr{s r set_streamrs*~ H##F+++++rcrt|}|dkr!tj|dSdS)zSet the current device. Args: device (torch.device or int): selected device. This function is a no-op if this argument is negative. rN)rr*r+%_accelerator_hooks_set_current_devicers r set_devicers=v & &F {{ 66v>>>>>{rc^tjt|dS)zReturn a dictionary of MTIA device properties Args: device (torch.device or int, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). TrZ)r*r+_mtia_getDevicePropertiesrrs rget_device_propertiesrrwrc8eZdZdZdefdZdZdededefdZdS) rzContext-manager that changes the selected device. Args: device (torch.device or int): device index to select. It's a no-op if this argument is a negative integer or ``None``. c@t|d|_d|_dS)NTrZr)ridxprev_idx)selfrs r__init__zdevice.__init__s!$Vd;;; rcXtj|j|_dSr")r*r+(_accelerator_hooks_maybe_exchange_devicerr)rs r __enter__zdevice.__enter__sII$(SS rtypevalue tracebackcXtj|j|_dS)NF)r*r+rrr)rrrrs r__exit__zdevice.__exit__s 8DDT]SSurN)rGrHrI__doc__rrrrrrrrr stsTTTSrc\eZdZUdZeded<dedfdZdZdeded efd Z d S) StreamContextaContext-manager that selects a given stream. All MTIA kernels queued within its context will be enqueued on a selected stream. Args: Stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: Streams are per-device. torch.mtia.Stream cur_streamr{cd|_||_tdd|_tjs|jd|_tjsdntjd|_ tjsdntjd|_ dS)NTr) rr{rrr*jit is_scriptingrPrasrc_prev_streamdst_prev_stream)rr{s rrzStreamContext.__init__.s $T400y%%'' x ..00 UDDej6O6OPT6U6U  ..00 UDDej6O6OPT6U6U rc|j}| |jdkrdStjd|_|jj|jkrUt |j5tj|j|_dddn #1swxYwYtj|dSNr) r{rr*rPr]rrrr)rrs rrzStreamContext.__enter__=s[  R F$z88>>   &**; ; ; )** T T',z'@'@AR'S'S$ T T T T T T T T T T T T T T T j)))))s$*BB!Brrrc|j}| |jdkrdS|jj|jkr$tj|jtj|jdSr)r{rrrr*rPrr)rrrrrs rrzStreamContext.__exit__Lsl[  R F   &**; ; ; J ! !$"6 7 7 7 d233333rN) rGrHrIrr__annotations__rrrrrrrrr s  ,----  x(;<      * * * 4S 4 4 4 4 4 4 4 4rrcHtj|||dS)zset stream specified by the stream id, device index and device type Args: stream_id (int): stream id in stream pool device_index (int): device index in topo device_type (int): enum device type N)r*r+_mtia_setStream) stream_id device_index device_types r_set_stream_by_idrZs$ HY kBBBBBrrc t|S)a,Wrap around the Context-manager StreamContext that selects a given stream. Arguments: stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: In eager mode stream is of type Stream class while in JIT it doesn't support torch.mtia.stream )rr~s rr{r{es   rrPc~tjdtdtjdgtj|S)zReturns the random number generator state as a ByteTensor. Args: device (torch.device or int, optional): The device to return the RNG state of. Default: ``'mtia'`` (i.e., ``torch.device('mtia')``, the current mtia device). z.get_rng_state is not implemented in torch.mtia stacklevelr )dtyper)warningswarn UserWarningr*zerosuint8rs r get_rng_staterpsC M8 ;s%+f = = ==r new_statec>tjdtddS)aSets the random number generator state. Args: new_state (torch.ByteTensor): The desired state device (torch.device or int, optional): The device to set the RNG state. Default: ``'mtia'`` (i.e., ``torch.device('mtia')``, the current mtia device). z.set_rng_state is not implemented in torch.mtiarrN)rrr)rrs r set_rng_staters0 M8r)*)r$rNr(rRrMrXr]ra memory_statsmax_memory_allocatedmemory_allocatedreset_peak_memory_statsrvrrirlrprzrrr{rrrrs MTIAGraphgraphgraph_pool_handle)rNr")rbrcr)T)rP)Lr threadingrcollections.abcrtypingrrrr*r_devicer torch._utilsr r r torch.typesr _utilsrr>int _device_tEventStreamrtupler+ Generatorrr&rlistlocalr5Lockr6r;r4rrrr r$r(boolr'r#r=r@r7rNrRrMrXr]raridictrlrprsrvrzrrrrrr{rrmemory mtia_graph__all__rrrrsI$$$$$$'''''''''' ++++++++EEEEEEEEEE%%%%%% '3# $    57E%(,c12777 t (2t8 d3i '(y%y~''%%'' 758+,,Hx4HHHHHH  758011H"X?HsHsHHHH 222 (((((0000f     I   $d$$$$ d22+2t2222 +c++++ <<<<< U U8I. U& U U U U U U8I. U& U U U UPQEE c]E,/EILE EEEE&+$sCx.++++ 7S#s+T127 77774 XX(9"5XsCxXXXX'''' ,v , , , , ?y ?T ? ? ? ?XX(9"5Xc3hXXXX(7474747474747474tCCC!8/0!]!!!! > >%S%, 67 >V > > > > @F$S#u|%;< "   r