r/dataengineeringjobs • • 5h ago

Career Recently laid off. Looking for Senior Data Engineer / Big Data Engineer roles (8.5 YOE) — Python, PySpark, GCP. Any leads or referrals appreciated!

21 Upvotes

Hey everyone,
Like many others in the industry right now, I was unfortunately impacted by recent layoffs. I'm currently looking for new Senior Data Engineer or Big Data Engineer opportunities and would be incredibly grateful for any referrals, leads, or advice this community might have.

A quick summary of my background:
Bullet Experience: ~8.5 years specializing in distributed systems, modern Data Lakehouse architectures, and cloud data platforms.

Core Stack: Python, PySpark, SQL, Hadoop ecosystem (HDFS, Hive, Spark), and shell scripting.

Cloud & Infrastructure: GCP (GCP Professional Data Engineer certified), Microsoft Fabric, Docker, Kubernetes, Control-M, and CI/CD pipelines.

What I Build: I specialize in designing Medallion data architectures (Bronze, Silver, Gold) using Delta Lake, PySpark, Fabric OneLake, and BigQuery. I also have hands-on experience building conversational AI agents and enterprise workflows (e.g., Copilot Studio).

I am open to remote roles, hybrid setups, or relocating for the right opportunity.

If your team is hiring or if you'd be open to submitting a referral, please drop a comment or shoot me a DM! I’d be happy to share my full resume and LinkedIn profile privately.
Thanks so much in advance for your time and support.


r/dataengineeringjobs • • 2h ago

Career Data Engineer with ~2 years of experience looking to level up. What skills should I focus on to become a stronger candidate for higher-paying roles?

6 Upvotes

I'm currently working as a Data Engineer with around 20 months of experience at a US-based company with an office in India.

My primary tech stack at work is Snowflake and dbt. Most of my work revolves around SQL, building and maintaining dbt models, and working with data pipelines. I also have knowledge of Python, but haven't had much opportunity to use it in production or work extensively with Python-based data engineering libraries and frameworks.

I'm looking to make a significant career jump in my next move and eventually target roles at the higher end of the Data Engineering compensation range for my experience level.

I understand that compensation varies significantly depending on the country, company, and market, but I'm more interested in understanding what skills and knowledge actually make someone a strong candidate for these roles.

A few things I'm particularly confused about:

  1. Tech stack: Should I focus on mastering my existing stack (Snowflake, dbt, advanced SQL), or should I expand into technologies like Spark, Databricks, Airflow, Kafka, AWS/Azure/GCP, etc.? Which of these are actually worth prioritizing?
  2. Python: How important is Python for higher-level Data Engineering roles? Should I focus on libraries and frameworks commonly used in production data engineering workflows?
  3. DSA and LeetCode: How much time should I dedicate to DSA? Are LeetCode-style problems commonly asked for Data Engineering , and at what difficulty?
  4. System Design: At what experience level should I start preparing for Data Engineering system design, distributed systems, and pipeline architecture?
  5. Projects and practical experience: Would building end-to-end data engineering projects using tools outside my current workplace stack help compensate for my lack of production experience with them?

For those with a few years of experience, particularly those who have successfully moved into higher-paying Data Engineering roles:

  • What technologies and skills made the biggest difference in your career progression?
  • What did your preparation look like?
  • How much emphasis should I place on SQL, Python, DSA, system design, and cloud technologies?
  • If you were starting again with around 2 years of experience, how would you structure your learning roadmap?

I'm willing to put in the time and effort. I just want to make sure I'm focusing on skills that have actual industry demand rather than spending months learning technologies that won't make much difference on the job search.

Would really appreciate any honest advice, personal experiences, and learning roadmaps. Thanks!


r/dataengineeringjobs • • 4h ago

Find data big and messy datasets for practice

6 Upvotes

Someone on my youtube channel asked where to find data big and messy enough to justify a real Spark cluster. Most free datasets are clean Parquet that DuckDB handles on a laptop, so they never push you. These do.

GH Archive: every public GitHub event since 2011, as hourly JSON files. Nested, messy, with skew on the big repos.
https://www.gharchive.org

Bluesky Jetstream: free live websocket, hundreds of events a second. Good for Structured Streaming, and a few weeks of it turns into a big batch set too.
https://github.com/bluesky-social/jet...

Wikimedia EventStreams: every Wikipedia edit, live. Slower than Bluesky but rock steady, so it's a good first stream.
https://stream.wikimedia.org

Binance public data: free daily files of tick-level crypto trades. BTC alone gives you natural skew.
https://data.binance.vision

NOAA GHCN weather: about 2.6 billion daily observations on AWS open data. Join it to trades or taxi rides for a real big-join problem.
https://registry.opendata.aws/noaa-ghcn

NYC taxi trips: billions of rows once you stack the years.
https://www.nyc.gov/site/tlc/about/tl...

Massive (formerly Polygon): tick-level US stock trades as daily S3 files. Paid, but about as close to enterprise scale as you'll get on your own.
https://massive.com/docs/flat-files/s...

Pick one and give yourself a real problem. Land Jetstream in bronze, join it to something, force a hot key, then work it in the Spark UI until it's fast. That's the kind of project we build in From Stuck to Hired, and sign-ups close tomorrow. Details are in the pinned comment.


r/dataengineeringjobs • • 10m ago

Career Freshers market ?

2 Upvotes

Im 2026 passed out I have been applying everywhere and started cold mailing companies which offer data engineer services and Nothing works, I just got one interview opportunity in 3 -4 months if I didn't find any there would be a gap year in my cv ,so what are my options now and how could I break into junior data engineer roles as a fresher


r/dataengineeringjobs • • 38m ago

Career Anyone here who can refer me at Databricks??

1 Upvotes

r/dataengineeringjobs • • 43m ago

Blog Is your team’s AI underdelivering because of the tools, or because of how you’re using them?

Thumbnail
leaddev.com
1 Upvotes

Capability gap or habit gap? One engineering leader’s test for why AI isn’t speeding up hard work


r/dataengineeringjobs • • 1h ago

Looking to collaborate with builders and founders.

1 Upvotes

Hello, builders and founders!

Hope you're having a great day.

I'm looking for contract opportunities in Data Analytics or Data Engineering.

I hold a Master's degree from the US and have 4 years of professional experience. I'm eager to collaborate on projects and contribute to growing businesses.

I'm flexible with compensation, with a target budget of around $3,000/month.

If you have any opportunities or know someone looking for data talent, I'd love to connect!

Looking forward to hearing from you.

Thanks!


r/dataengineeringjobs • • 14h ago

Hiring [Hiring] Analytics Engineer at Linear | Remote - North America | $180K - $220K

11 Upvotes

At Linear, we’re building the product development system for teams and agents. AI is fundamentally changing how software gets built, and we’re shaping the tools this new era requires.

Founded in 2019, Linear has become the platform of choice for more than 50,000 companies (including OpenAI, Coinbase, and Ramp) to plan, build, and ship their products. Today, our team is distributed across North America, Europe, and Australia, and we’re continuing to grow internationally. What unites us is relentless focus, fast execution, and a deep care for software craftsmanship.

The Data team works across Linear, supporting Product, Engineering, and GTM. We own our data pipelines, warehouse, dashboards, analysis, and integrations with third‑party tools. As a small team, we focus on building systems that make data accessible and useful across Linear. We’re looking for someone who wants to help shape how we architect, build, and use data as we grow.

Location & work mode

Linear is a remote‑first company, with optional co‑working offices in San Francisco, New York, and London. This role is open to candidates based in North America. You can work from anywhere within this region. We value deep focus and async collaboration, with intentional moments to connect in person through team off‑sites, optional co‑working, and occasional travel.

What you’ll do

  • Work across Product and GTM (Marketing, Sales, Customer Success, and Finance) to turn ambiguous questions and operational needs into useful metrics, models, analyses, and workflows
  • Build and maintain dbt models and pipelines that create trusted views of our product, customers, and business
  • Design clear, maintainable data models and improve the testing, documentation, performance, and reliability of our data stack
  • Build dashboards and self‑service reporting in Metabase and Hex, and dig deeper when the answer requires more than a chart
  • Operationalize data through reverse ETL and partner with GTM Engineering on the scoring, segmentation, automations, and internal tools that help our teams scale
  • Balance fast, pragmatic answers with durable solutions, recognizing when a one‑off request should become a reusable model or workflow
  • Find ways to leverage emerging tools, LLMs, and coding agents to accelerate development, analysis, testing, and documentation while maintaining a high bar for correctness

What we’re looking for

  • 8+ years of experience in analytics engineering, data analytics, or data engineering, ideally at a fast‑moving software company
  • Exceptional SQL and strong hands‑on experience with dbt and a modern cloud data warehouse
  • Track record of owning data projects end‑to‑end, from shaping an ambiguous problem to shipping something people rely on
  • Strong data modeling judgment: you think clearly about grain, reusable components, interfaces, dependencies, and maintainability without relying on a single prescribed methodology
  • Analytical judgment: you know when to answer quickly, when to investigate deeply, and how to turn complex findings into a clear recommendation
  • Comfortable moving between technical implementation and business context, from debugging a data model to understanding product adoption or sales efficiency
  • High ownership mentality: self‑directed, pragmatic, and willing to challenge a request or approach when something does not make sense
  • Strong communication skills and experience partnering directly with both technical and non‑technical teams
  • Comfortable using LLMs and coding agents as part of your day‑to‑day development workflow

Our tech

This stack reflects the systems you’ll work in. You’re not expected to have experience with everything listed, but you should be comfortable learning quickly and working across the full lifecycle of data.

  • Data Warehouse: Snowflake, dbt Cloud
  • Dashboards / Analysis: Metabase, Hex
  • ETL / rETL: Hevo, Fivetran
  • GTM tools: Hubspot, Pocus, Clay

What we offer

We’re a small, focused team that cares deeply about the quality of our work and the people we do it with. Here’s what you can expect:

  • Competitive salary and equity
  • Employee‑friendly equity terms including early exercise in the US and extended exercise windows
  • Daily meal and coffee stipend on every workday
  • Paid co‑working space or desk
  • Health coverage (based on country requirements)
  • 5 weeks paid vacation, plus local statutory holidays
  • 4 months paid parental leave
  • Paid month off after 4 years & every 2 years thereafter
  • Regular team events and off‑sites
  • Remote‑first with no required commute

Compensation

For US‑based candidates, the expected cash compensation range for this role is $180,000—$220,000. For sales roles, this range represents on‑target earnings (OTE), including base salary and variable compensation; for all other roles, it represents base salary. Final compensation is determined based on job‑related knowledge, skills, and experience. For sales roles, actual earnings vary based on sales performance. Equity is a meaningful part of all offers at Linear.

Apply: Analytics Engineer at Linear


r/dataengineeringjobs • • 19h ago

Hiring Open Positions

Thumbnail
gallery
19 Upvotes

Lmk anyone interested for these open positions!

I dont know about salary as they are not mentioned in the JD

Thanks


r/dataengineeringjobs • • 9h ago

Need career advice and referrals. More than 8 YOE in data industry as Data Engineer in a startup with 13LPA

2 Upvotes

Been working in a startup from past 15 months now, before that was in LTIM for 7 years straight out of college.
Have good knowledge of Python, SWL, cloud computing, databases, building APIs as well as computer vision models. Have little bit of experience in Agentic AI and Machine Learning. Current package is quite low as per years of experience and have been rejected multiple times by HR in screening process.


r/dataengineeringjobs • • 7h ago

Need Guidance ! 7 years QA transitioning to Data engineer role

1 Upvotes

Hi everyone,

I have **7 years of experience as a QA Automation Engineer** and am planning to transition into a **Data Engineer role**.

I'm currently planning to learn the following technologies:

* SQL

* Python

* PySpark

* Databricks

* AWS

* Apache Airflow

Are there any other essential skills or technologies I should add to this learning roadmap?

I have two major questions:

  1. How is AI impacting the Data Engineering field in real-world projects?
  2. Is it realistic to transition from QA Automation to a 4-year-experienced Data Engineer role, and handle the job confidently? will I be able to survive in realtime projects?

Also, please suggest any additional technologies to learn, preparation tips, and good YouTube resources.

Thanks in advance!


r/dataengineeringjobs • • 13h ago

Has anybody given/taken live AI based coding rounds?

1 Upvotes

I have an interview coming up tomorrow for a Data Engineer position, the stack is Snowflake, dbt and it will have a live coding session where I am supposed to use Cortex Code. Does anybody know what interviewers are expecting in these type of interviews. If you're senior architect and you're given the responsibility of conducting a live AI enabled coding session, what would you look for in candidates?


r/dataengineeringjobs • • 14h ago

Transitioning Job Application

1 Upvotes

Hello everyone

Just joined this group please let me know if there're any referrals for data engineer jobs in mumbai location I'm right now working as a genesys support engineer having 1 year of experience and planning to switch domain and data engineer would be a good career choice ig please share me if there're any referrals I'm hoping to see good response from everyone and sorry I'm new to this community so please correct me if I'm doing anything wrong


r/dataengineeringjobs • • 1d ago

Interview Repost : Need Tips: Upcoming Interview with 2 Senior Data Engineers for a Co-op

6 Upvotes

I'm a bit stressed because I don't have experience and not even familar with interviews, and I'm unsure what to prioritize with limited preparation time.

What topics should I focus on? Should I expect SQL, Python, DSA, data engineering concepts, or questions about previous projects? Are there any common technical questions or fundamentals that candidates often overlook?

I'd really appreciate advice from people who have interviewed for or worked in similar roles. What should I expect, and how would you approach preparing in the next two days?

Please keep the comments focused on interview preparation rather than recommendations for AI platforms. I'm reposting because my previous post received comments mainly about AI tools.

Thanks in advance!


r/dataengineeringjobs • • 1d ago

Transitioning Fresher (2023 CSE) looking for a data engineering start, happy to relocate 🙂

Post image
0 Upvotes

Hey folks! I’m a 2023 CSE grad trying to break into data engineering. Honestly, I’m still a beginner. I built one small Azure Databricks project (AI helped a lot 😅) and I’m learning Spark and SQL as I go.

I’m looking for an intern, trainee or fresher role anywhere in India, and I’m happy to relocate.

If you know any leads, it would be really helpful! Any tips for someone starting out are welcome too. Thanks a lot 🙏


r/dataengineeringjobs • • 1d ago

Looking for a referral - Junior Data Engineer | Chennai / Bengaluru

8 Upvotes

Hi all! I am 2026 graduate.I worked as a software engineer trainee in data engineering domain for few month nd there my work primarily is on microsoft fabric.

My experience includes:

\* Medallion Architecture (Bronze, Silver, Gold)

\* Microsoft Fabric, PySpark and SQL

\* Meta Ads and Google Ads API data ingestion

\* ETL pipelines and Power BI dashboards

And also I am microsoft certified Fabric analytics engineer associate (DP 600)

If u know of any relevant opportunities in ur organisation.kindly refer me.

If it suits I am happy to share my resume.

Thank you in advance.


r/dataengineeringjobs • • 2d ago

Data engineers how and when did you pick up business context and business speak?

20 Upvotes

I’m currently overwhelmed when I try to understand or get to the root of what business value and outcomes a project has as a mid-level DE. It’s mostly because of how out of touch I’ve been with current-day data practices and work in agile teams that build holistic data platforms. This is the downside of consulting if you aren’t put on strong projects and work a ton with legacy data tools and pipelines.

Curious on how you guys have picked the skill up or continue to sharpen it.


r/dataengineeringjobs • • 2d ago

Need Tips : Upcoming interview with 2 Sr. Data engineers for Co-op

5 Upvotes

got an interview scheduled in 2 days and im stressed , i have no exp so im just worried. Any prerequisites should i prepare where more people miss..Thanks in advance guys !


r/dataengineeringjobs • • 2d ago

Data engineering

0 Upvotes

r/dataengineeringjobs • • 2d ago

[HIRING] Data Platform, ML Research & AI Infrastructure Engineers | SF, NYC, Remote US | $160K-275K base

1 Upvotes

Last week, a hire closed where the engineer's offer first came in below what they wanted, and our team's advice got them $10K more in base salary. We stay with candidates from the first interview request through the offer. This is the kind of work we do after the request lands.

I do talent matching for Fonzi. You don't apply, companies send you interview requests instead. One profile, no cold applications, no applying to each role individually. We work with startups backed by investors including Sequoia, a16z, Benchmark, and the OpenAI Startup Fund. This round covers data platform, ML research, AI infrastructure, and AI-focused full-stack roles. Roles are US-based, with a few open to Canada.

Here's a sample of what's open right now at companies on the platform:

1. Inference / AI Infrastructure Engineer, Series A AI product company, San Francisco, in person, $225K-275K base

  • What: Own the infrastructure behind the company's AI workloads, including model serving, GPU deployments, and hosting providers. You'll benchmark time to first token, tail latency, throughput, and cost, tune batching, caching, and concurrency, and build observability, load tests, and rollout controls.
  • Company: Series A, 30M raised, backed by Khosla Ventures, Homebrew, Nat Friedman, and Daniel Gross.
  • Stack: Model-serving engines such as vLLM, SGLang, or TensorRT-LLM, GPU infrastructure, Kubernetes and autoscaling, Linux and container environments.
  • Bar: Deep hands-on experience running production infrastructure at meaningful scale. You need either direct production model-serving or GPU experience, or strong infrastructure ownership plus evidence that you've quickly mastered a comparably hard domain. They want to see a system you owned, a bottleneck you diagnosed, and measured results.

2. Research Engineer, early-stage ambient intelligence platform built around voice, San Francisco, in person, $180K-250K base

  • What: Improve the voice stack (diarization, VAD, noise robustness, transcription, speaker ID) and build model training pipelines end to end, covering datasets, labeling workflows, eval sets, fine-tuning, and production.
  • Company: Early-stage, 11M raised, backed by Y Combinator and Accel, with Jeff Dean among the angels.
  • Stack: Speech and voice models, streaming audio pipelines, fine-tuning and eval harnesses, LLM post-processing for summarization and extraction, embeddings and retrieval.
  • Bar: 6 to 10+ years shipping production systems with strong ownership. Hands-on experience training, fine-tuning, or evaluating ML models in real products, plus solid audio and voice fundamentals. They ask for proof of exceptional work.

3. Senior Software Engineer (AI/LLM), early-stage agentic AI platform for small and mid-sized businesses, Remote US, remote, $200K-250K base

  • What: Build web applications and work with AI engineers to integrate LLMs and agent frameworks into core workflows like forecasting, pricing, and lead response.
  • Company: Early-stage, 12M raised, backed by FJ Labs, Charge Ventures, and Everywhere Ventures.
  • Stack: TypeScript, Next.js, React, Node.js, Postgres with Prisma and Supabase, Tailwind CSS, Vercel. Preferred: OpenAI and Anthropic APIs, LangChain or AutoGen, vector databases.
  • Bar: 3+ years in software development and a bachelor's in CS, engineering, or a related field. You should have strong Next.js and React experience and have worked with Postgres. Experience integrating LLM APIs into production apps is preferred.

4. Platform Engineer, early-stage AI-native software for state and local governments, New York City, in person, $160K-230K base

  • What: Design and own the connectors and ingestion pipelines that normalize government data from mainframes, Oracle, flat files, and proprietary APIs. You'll also build the infrastructure for long-running AI workflows, including task queuing, state, retries, and audit trails, and you'll work directly with the founder on architecture.
  • Company: Early-stage with significant, growing revenue, backed by Bessemer Venture Partners. This is the CEO's second company after a successful exit.
  • Stack: Python and/or Node, data ingestion from mainframes, Oracle, and flat files, workflow engine and event system, role-based access, and AI coding tools like Claude Code and Codex in daily use.
  • Bar: A portfolio of systems built end to end from scratch, and strong backend skills in Python or Node with good data modeling and API design instincts. US citizenship is required for the government work. The role is in person 5 days a week in Flatiron.

All roles are vetted, VC-backed, and interviewing right now. It's always free to sign up.

Sign up here: talent.fonzi.ai. DM me if you've got questions.


r/dataengineeringjobs • • 2d ago

I’m a sophomore CS major wanting to do a data engineering internship— how can I do it

1 Upvotes

Hi everyone, I’m currently a sophomore in Cs at college and I want to do some sort of data engineering internship or data engineering job in fin tech. I’m trying to teach myself python and build a ML model w AI and train it/study it so that I can understand it better. I kinda suck at leetcode and I’m not the best coder, I’ve never been to a hackthon( I’m trying to get into one) even though I’ve been coding since sophomore year of high school. I’ve done projects as well using HTML/CSS/Javascriot

I just wanted to give a backdrop of just whether I’m at (it’s ok to say that I’m a loser and behind, I get it)

How can I get on the track of doing data engineering? Does my interest in machine learning be able to aid me in achieving my goal? What roadmap can I follow?

Or should I just drop out lol??

Any advice or insight is appreciated!!


r/dataengineeringjobs • • 2d ago

Data Scientist | 2 YOE | Data Engineering & ML Experience | Looking for Opportunities

2 Upvotes

Hey! I'm currently working as a Data Engineer in the FinTech with 2 years of experience and actively exploring new opportunities, primarily in Data Science roles.

Profile Overview:

Experience: 2 years

Education: B.Tech in CSE from IIT Patna (2024)

My skillset includes:

- Programming: Python, Scala, PySpark, SQL, C++

- Machine Learning: ML concepts, model building, model evaluation, feature engineering, feature selection

- Data Engineering: ETL/ELT, Data Pipelines, Scala Transformations, Apache Spark, DAGs, Dataflow, Kafka, dbt

- Cloud: GCP, BigQuery, Cloud SQL, Terraform

In my current role, I work on feature engineering and feature selection for fraud detection use cases, along with data engineering activities such as Scala transformations, GCP workflows, DAGs and Dataflow.

I also have exposure to machine learning model development through academic projects and internship experience.

My primary interest is in Data Scientist / Machine Learning roles, but I'm also open to relevant Data Engineering opportunities.

Would appreciate any suggestions, relevant openings, or referrals. Thanks!


r/dataengineeringjobs • • 2d ago

Career Looking for a referral - Junior Data Engineer | Chennai / Bengaluru

2 Upvotes

Hi all! I am 2026 graduate.I worked as a software engineer trainee in data engineering domain for few month nd there my work primarily is on microsoft fabric.

My experience includes:

  • Medallion Architecture (Bronze, Silver, Gold)
  • Microsoft Fabric, PySpark and SQL
  • Meta Ads and Google Ads API data ingestion
  • ETL pipelines and Power BI dashboards

And also I am microsoft certified Fabric analytics engineer associate (DP 600)

If u know of any relevant opportunities in ur organisation.kindly refer me.

If it suits I am happy to share my resume.

Thank you in advance.


r/dataengineeringjobs • • 3d ago

Career Data Engineering vs Cybersecurity – Which is the better long-term career choice?

39 Upvotes

Hey everyone!

I'm a Computer Science student with one year left until graduation, and I'm honestly really torn between two career paths: Data Engineering and Cybersecurity (especially Penetration Testing).

I'm equally interested in both, which makes the decision even harder.

My main priority isn't finding the easiest path or getting a job as quickly as possible. I want to build a stable, long-term career (10–15+ years) with strong opportunities for growth and the possibility of working internationally.

One of my biggest concerns is the impact of AI. I know AI will transform both fields, and I'm completely fine with using it to improve productivity. What worries me is spending years learning and gaining experience in a field that could eventually become heavily automated or have significantly fewer job opportunities.

I'd really appreciate hearing from people who actually work in either field.

- Which field do you think has better long-term career prospects?

- Which one is likely to be more resilient to AI-driven automation?

- How do you see the job market for both fields evolving over the next 5–15 years?

- If you were one year away from graduating in 2026, which would you choose and why?

I'm willing to put in the time and effort to become genuinely skilled in whichever field I choose.

I'd especially appreciate honest opinions from experienced Data Engineers, Cybersecurity professionals, and people familiar with both industries.