9i* &ddlZddlmZddlmZddlmZddlmZm Z m Z ddl m Z m Z ddlZddlmZddlmZddlmZe d d Ze d ZddZeZeZGddZe ddd Ze dejeejZGdde e eeefZefde edeeeeefefgeeeefeefffdZ!dejde"e#efdzfdZ$dS)N) OrderedDict)Callable)wraps) ConcatenateGenericProtocol) ParamSpecTypeVar)_State)_get_root_modules_TT) covariant_P__composable_api_state_keycL|dttjS)N_)struuiduuid4)strings Y/var/www/icac/venv/lib/python3.11/site-packages/torch/distributed/_composable/contract.pygenerate_state_keyrs$ * *s4:<<(( * **ceZdZdS) RegistryItemN)__name__ __module__ __qualname__rrrrsDrr_TStater )boundr_McPeZdZdejdejdefdZdej de fdZ dS) _ContractFnargskwargsreturncdSNr)selfr%r&s r__call__z_ContractFn.__call__(rmodulecdSr)r)r*r-s rstatez_ContractFn.state*r,rN) rrrrr%r&r r+nnModuler r/rrrr$r$'sGFbgFFrFFFF6BI6'666666rr$ state_clsr'ctdttttftfdt tttftt fffd }|S)a% Decorate a function as a composable distributed API, where the first argument of the function must be an :class:`nn.Module` instance or sequence of :class:`nn.Module` instances. The decorator verifies that the decorated function does not modify fully-qualified names (FQNs) for parameters, buffers, or modules. The decorated function can return different module instances than the input modules; the FQN invariant will be enforced following the input order. When a function ``func`` is decorated by ``@contract()``, a ``.state(module: nn.Module)`` method will be installed to the decorated function. Then you can retrieve and modify the state on a module by calling ``func.state(module)``. Example:: >>> # xdoctest: +SKIP >>> import torch.nn as nn >>> >>> class MyModel(nn.Module): >>> def __init__(self) -> None: >>> super().__init__() >>> self.l1 = nn.Linear(10, 10) >>> self.l2 = nn.Linear(10, 10) >>> >>> def forward(self, x): >>> return self.l2(self.l1(x)) >>> >>> @contract() >>> def my_feature(module: nn.Module) -> nn.Module: >>> my_feature.state(module).some_state = "any value" >>> return module >>> >>> model = MyModel() >>> my_feature(model.l1) >>> assert my_feature.state(model.l1).some_state == "any value" >>> my_feature(model.l2) >>> model(torch.randn(2, 10)).sum().backward() funcr'c tdtdtjdtjdtffd }dt jdtffd }||_|S)Nr-r%r&r'c |}t|tjr|g}ntt |}}t }g}g}g} |D]}t } t } |jt| } t| tstd| |jt| } t| tstd| | vs j | vrtdj d|| || j ||t ||t || t ||g|Ri|}||}t|tjr|g}ntt |}g}g}g}|D]}|t ||t ||t |t%| }t%|}||krtj d|d|ddtt&d tt&d t&fd }t)||D]P\}}|t |t |d Qt)||D]P\}}|t |t |d Qt)| |D]P\}}|t |t |dQ|S)Nz-Distributed composable API states corrupted: z/Distributed composable API registry corrupted: zOEach distinct composable distributed API can only be applied to a module once. z3 has already been applied to the following module: zB should return the same number of modules as input modulesInputs: z modules Outputs: z modules orig_fqnsnew_fqns check_keyc||krdSt|t|}}||z }||z }t|st|rt|d|d|t|d|d|)NzVComposable distributed API implementations cannot modify FQNs. FQNs only in original: z FQNs only in new: z[Composable distributed API implementations cannot modify the order of FQNs. Original FQNs: z New FQNs: )setlen RuntimeError)r7r8r9 orig_fqn_set new_fqn_set orig_onlynew_onlys r check_fqnz;contract..inner..wrapper..check_fqns((F,/ NNCMMk (;6 &5y>>S]]&$882;88.688'$00+400&. 00rzChecking parameters: zChecking buffers: zChecking modules: ) isinstancer0r1r listrr__dict__ setdefault STATE_KEYdictAssertionError REGISTRY_KEYrappendnamed_parameters named_buffers named_modulesr<rzipkeys)r-r%r& inp_modulemodulesr/ registry_itemall_orig_named_paramsall_orig_named_buffersall_orig_named_modulesdefault_all_statedefault_registry all_stateregistryupdatedupdated_modulesall_new_named_paramsall_new_named_buffersall_new_named_modulesnum_orig_modulesnum_new_modulesrBorig_named_paramsnew_named_paramsorig_named_buffersnew_named_buffersorig_named_modulesnew_named_modulesr4r2s rwrapperz(contract..inner..wrapper`s!  J&"),, :!( ,DLL99IKKE(NNM DF !DF "AC "" S S.inner..get_states0?--c$ii r) rr"rr%r&r0r1r r/)r4rhrkr2s` rinnerzcontract..inner\s tM M 7M iM  M M M M M M  M ^ bi F       " r)rrrr"rr$r )r2rls` rcontractrm-s^ 9[{2r6*B./[ [R("g5 6[[[[[[z Lrr-c.t|tdS)z Get an ``OrderedDict`` of composable APIs that have been applied to the ``module``, indexed by the API name. If no API has been applied, then this returns ``None``. N)getattrrJ)r-s r _get_registryrps 6< . ..r)r)%r collectionsrcollections.abcr functoolsrtypingrrrtyping_extensionsr r torchtorch.nnr0#torch.distributed._composable_stater torch.distributed.utilsr r rrrGrJrr r1rDr"r$typermrHrrprrrr{sR ######$$$$$$111111111100000000 666666555555 WTT"""Yt__++++   !!##          ')8t < < < WT29d29o..77777(GBGO4777 &MMG}M  k"b&!2%&' BF#R013MMMM`/")/S,->(?$(F//////r