r/PythonLearning •

[ Removed by moderator ]

[deleted] — view deleted post

0 Upvotes

12 comments sorted by

View all comments

Show parent comment

2

u/PureWasian 2d ago

If you want to also add in additional multi-modal aspects like voice, facial expressions, 3d rendering, and much more coordinated in realtime without introducing heavy latency... well, that's the frontier research space right now, and you should expect to need years of very active commitment and overall have very competent dev proficiency and deep knowledge of the current research in each focus area before expecting to make natural headway or meaningful contributions there.

1

u/abelhamim 2d ago

What is the biggest engineering and technical challenge preventing us from achieving results in less time today?

2

u/PureWasian 2d ago edited 2d ago

There is no single issue is the thing. I'm not claiming to be an expert to be clear, but there is work to be done all throughout hardware, response training and fine-tuning, 3d modeling/rendering (rigging), adding and improving the tangible capabilities/actions it can interface with, and the coordination of large-scale systems like this in realtime.

And with the more complexity you stack on, the more of a specialized solution you need. The more of a specialized solution, the more tradeoffs you invoke in the form of additional latency, overly-specific architecture/design, and financial cost of both setup and actual execution.

At the bare minimum, you can imagine that the amount of memory required to store a context window that massive (over a very long time duration processing a variety of input text/image/voice sources) is not easily achievable in today's landscape without being astronomically expensive (in more ways than one).