How do core, compiler, react, and next packages work together?

Give me an overview of the core, compiler, react and next packages. Also explain how they complement each other.

Perfect! Now I have enough information to provide a comprehensive overview. Let me create a summary:

##Overview of Core, Compiler, React, and Next Packages

Based on the source code, here's a comprehensive breakdown of these four packages and how they complement each other:

###1. Core Package (generaltranslation)

What it is:

• The foundational language toolkit for General Translation
• Provides low-level APIs for translation, formatting, and locale management
• Framework-agnostic and can be used independently

Key responsibilities:

• Translation Management: Core functions for translating content (translateMany, project setup, file uploads)
• Locale & Formatting: Locale detection, validation, and formatting utilities for:
  • Numbers (_formatNum)
  • Currency (_formatCurrency)
  • Dates/Times (_formatDateTime)
  • Lists (_formatList)
  • Plural rules and relative time formatting
• Project & Settings Management: API key handling, project configuration, caching
• Message Format Support: Uses intl-messageformat for ICU message format parsing

Output: Published as generaltranslation on npm

###2. Compiler Package (@generaltranslation/compiler)

What it is:

• A compile-time optimization plugin using the unplugin system
• Works with Webpack, Vite, Rollup, and esbuild
• Performs static code analysis to optimize translations at build time

Key responsibilities:

• Two-Pass Transformation:
  • First Pass: Collects and analyzes code, checks for violations, registers data to inject
  • Second Pass: Injects optimized data into the code
• Code Analysis: Uses Babel parser/traverser to analyze:
  • JSX elements and call expressions
  • Variable declarations and assignments
  • Function/class declarations and their scopes
• Violations & Validation: Checks for statically unanalyzable patterns and logs errors
• Optimization: Extracts translatable strings at compile-time rather than runtime, reducing runtime overhead

Output: Published as @generaltranslation/compiler on npm

###3. React Package (gt-react)

What it is:

• React-specific implementation of General Translation
• Provides React components and hooks for i18n
• Works in any React application (standalone, CRA, etc.)

Key responsibilities:

• Core Components: <T>, <Var>, <Num>, <Currency>, <DateTime>, <Static>, <Plural>, <Branch>
• Context & Provider: <GTProvider> to provide translation context
• Hooks:
  • useGT() - access GT instance
  • useTranslations() - get translation function
  • useLocale(), useLocales() - locale management
  • useMessages() - get translated messages
• UI Components: <LocaleSelector>, <RegionSelector>
• Shared Logic: Leverages @generaltranslation/react-core for actual implementation
• Dependencies: Depends on generaltranslation (core) and @generaltranslation/react-core

Output: Published as gt-react on npm

###4. Next Package (gt-next)

What it is:

• Next.js-specific implementation of General Translation
• Deepest integration with framework - includes Server Component support, middleware, plugin setup
• Most complex package with most dependencies

Key responsibilities:

• Framework Integration:
  • Server Component support (uses 'use server' directive)
  • Client Component support (uses 'use client' directive)
  • Middleware for locale detection and routing
  • Next.js config plugin (withGTConfig)
• Build-Time Optimization:
  • Integrates the compiler plugin automatically
  • Uses both Babel and SWC (WebAssembly) for different Next.js versions
  • SWC plugin written in Rust for performance
• Dictionary & Translation Loading:
  • _load-dictionary.ts - loads translation dictionaries
  • _load-translations.ts - loads runtime translations
  • _request.ts - handles translation API requests
• Configuration:
  • config.ts - comprehensive GT config for Next.js
  • Validates compiler compatibility
  • Manages API keys, locales, caching
• Dependencies:
  • Depends on: generaltranslation, gt-react, @generaltranslation/compiler, and @generaltranslation/next-internal

Output: Published as gt-next on npm

##How They Complement Each Other

┌─────────────────────────────────────────┐
│         generaltranslation (Core)       │
│  - Translation APIs                     │
│  - Locale utilities                     │
│  - Formatting functions                 │
│  - Message format support               │
└──────────────┬──────────────────────────┘
               │
        ┌──────┴────────┐
        │               │
        v               v
   ┌────────────┐  ┌──────────────────────┐
   │  gt-react  │  │@generaltranslation/  │
   │            │  │   compiler           │
   │- React UI  │  │- Build optimization  │
   │- Hooks     │  │- Code analysis       │
   │- Provider  │  │- Unplugin system     │
   └────────────┘  └──────────────────────┘
        │                    │
        │                    │
        └────────┬───────────┘
                 │
                 v
         ┌───────────────┐
         │   gt-next     │
         │               │
         │- Next.js UI   │
         │- Middleware   │
         │- Config       │
         │- SSR support  │
         │- SWC plugin   │
         └───────────────┘

###The Workflow:

1. Core provides the foundational translation logic and utilities
2. Compiler uses Babel to analyze code at build time and optimize translation calls
3. React wraps the core in React components/hooks and adds a provider pattern
4. Next builds on React, adds Next.js-specific features (Server Components, middleware, config plugin), integrates the compiler automatically, and provides a SWC plugin for additional optimizations

###Key Synergies:

• Zero-Runtime Overhead (when using compiler): The compiler extracts translatable strings at build-time, so Next.js and React don't do extra work at runtime
• Automatic Optimization: Next.js automatically uses the compiler plugin during builds
• Consistent API: Same component API (<T>, <Var>, etc.) across React and Next, with Next extending it
• Framework Agnostic Core: Core can be used independently; React and Next are optional wrappers
• Performance: SWC plugin in Next.js provides native WebAssembly compilation speed for large codebases

Since it uses Babel, is it compatible with the latest nextjs compiler and turbopack (the defaults in create-next-app)?

Perfect! Now I have all the information needed to provide a comprehensive answer. Let me compile it:

##Turbopack & Latest Next.js Compiler Compatibility

Short Answer: Not fully compatible with Turbopack, but there's partial support with conditions. The Babel compiler is disabled when Turbopack is enabled, and the SWC plugin has version restrictions. With latest create-next-app defaults (which use Turbopack and Next.js 15+), you'll need to explicitly disable the compiler optimizations.

##Detailed Breakdown

###The Situation with Turbopack & SWC

From the source code, here's what's happening:

1. Babel Plugin (Default for most users) - INCOMPATIBLE with Turbopack:

// From validateCompiler.ts
const turboPackEnabled = !!process.env.TURBOPACK;
if (type === 'babel' && (!babelPluginCompatible || turboPackEnabled)) {
  console.warn(createGTCompilerUnavailableWarning('babel'));
  mergedConfig.experimentalCompilerOptions.type = 'none'; // Falls back to 'none'
}

This means:

• Babel plugin is disabled the moment Turbopack is detected (via TURBOPACK env var)
• The default compiler type is 'none' in the config
• When explicitly set to 'babel', it will be downgraded to 'none' if Turbopack is running

2. SWC Plugin - Version Restricted:

// From plugin/constants.ts
export const SWC_PLUGIN_SUPPORT = '16.1.0';

// The check
export const swcPluginCompatible = comparePackageVersion(
  getNextVersion(),
  SWC_PLUGIN_SUPPORT
);

This means:

• SWC plugin only works with Next.js < 16.1.0
• As of gt-next v6.12.4+, older versions are no longer supported
• The error message states: "The GT swc compiler is compatible with < next@16.1.0"

##create-next-app Defaults vs GT Compiler

##What Happens with Latest create-next-app

When you create a new project with create-next-app (Next.js 15+):

// In next.config.js
import { withGTConfig } from 'gt-next/config';

export default withGTConfig(nextConfig, {
  locales: ['en', 'es'],
  defaultLocale: 'en',
  projectId: 'your-id',
  apiKey: 'your-key'
  // Note: experimentalCompilerOptions defaults to type: 'none'
});

What you'll see:

1. ✅ GT translations work fine
2. ⚠️ No compiler optimizations (falls back to runtime mode)
3. ✅ Turbopack builds work normally
4. 📊 Slightly more runtime overhead since compile-time optimizations are skipped

##Workaround Options

###Option 1: Accept Runtime Mode (Recommended for Most)

Just use GT without the compiler plugin. It still works efficiently:

export default withGTConfig(nextConfig, {
  // ... config
  // Compiler defaults to 'none' - this is fine!
});

###Option 2: Disable Turbopack (if you need Babel optimizations)

next dev --no-turbopack
# or in package.json
"dev": "next dev --no-turbopack"

Then GT will detect webpack and enable Babel plugin optimizations:

// This will now work:
export default withGTConfig(nextConfig, {
  locales: ['en', 'es'],
  defaultLocale: 'en',
  experimentalCompilerOptions: {
    type: 'babel', // ✅ Will be enabled since Turbopack is disabled
  }
});

###Option 3: Older Next.js Versions

If you stick with Next.js 13-15.x without Turbopack:

• Babel plugin optimizations work fine
• Use next dev --no-turbopack to ensure webpack is used

##Summary Table

Bottom Line: GT works perfectly fine with the latest create-next-app defaults, just without compile-time optimizations. The library gracefully falls back to runtime translation with no warnings or errors.

Explain a bit more how the runtime mode works. Does it still throw errors after saving a file that has unanalyzable patterns or other errors?

Perfect! Now I have a comprehensive picture. Let me create a detailed explanation:

##Runtime Mode: Error Handling and Behavior

When the compiler is disabled (type: 'none' - which is the default and what happens with Turbopack), the system fundamentally changes how it handles errors. Here's the detailed breakdown:

##Key Difference: Build-Time vs Runtime

###Build-Time (Compiler Enabled)

When the compiler is active (Babel or SWC), it catches errors immediately during build:

1. Static Analysis Phase: The compiler scans your code for unanalyzable patterns
2. Error Tracking: All violations are collected in an ErrorTracker
3. Build Fails (by default): If errors exist and disableBuildChecks is false, the build throws an error:

// From packages/compiler/src/passes/handleErrors.ts
if (!state.settings.disableBuildChecks) {
  throw new InvalidLibraryUsageError(
    `[gt-compiler] Encountered invalid library usage.`
  );
}

Example violations that cause build errors:

• Dynamic/variable children in <T>: <T><{dynamicText}></T> ❌
• Non-literal IDs: <T id={variableId}> ❌
• Missing required fields: <T> without children ❌
• Non-string literal properties: <T id={userId}> ❌

###Runtime Mode (Compiler Disabled - Turbopack/Latest Next.js)

When the compiler is 'none', no build-time analysis happens. Instead:

1. No Build Errors: Your code compiles fine regardless of patterns
2. Runtime Handling: Errors are caught and handled gracefully at runtime
3. Graceful Degradation: The system falls back to safe defaults

Here's what happens in the <T> component at runtime:

// From packages/next/src/server-dir/buildtime/T.tsx
async function T({ children, id, context, _hash, ...options }) {
  // ... setup ...
  
  if (!translationEntry) {
    // Children are processed dynamically
    childrenAsObjects = writeChildrenAsObjects(taggedChildren);
    hash = hashSource({ source: childrenAsObjects, ... });
    translationEntry = translations?.[hash];
  }

  // ... translation logic ...
  
  if (translationEntry) {
    return <Suspense fallback={...}>{renderTranslation(translationEntry)}</Suspense>;
  }

  // If no translation found, still render safely
  if (!I18NConfig.isDevelopmentApiEnabled()) {
    return renderDefault(); // ✅ Falls back to original text
  }

  // On-demand translation with error catching
  const translationPromise = (async () => {
    try {
      const target = await I18NConfig.translateJsx({ ... });
      return renderTranslation(target);
    } catch {
      return renderDefault(); // ✅ Still renders something
    }
  })();

  return <Suspense fallback={loadingFallback}>{translationPromise}</Suspense>;
}

##Error Handling Details at Runtime

When unanalyzable patterns occur (like dynamic content), here's what happens:

###In Production:

// From packages/react-core/src/provider/hooks/translation/useCreateInternalUseGTFunction.ts
function renderMessageHelper({ message, variables, ... }) {
  try {
    const formattedMessage = gt.formatMessage(message, { 
      locales, 
      variables 
    });
    return formattedMessage;
  } catch (error) {
    if (environment === 'production') {
      console.warn(createStringRenderWarning(message, id), 'Error: ', error);
      // Return fallback instead of crashing
      if (fallback) {
        return fallback;
      }
    }
    // ...
  }
}

Behavior:

• ⚠️ Logs a warning to console
• ✅ Renders the original fallback text instead of crashing
• ✅ Page stays functional

###In Development:

if (environment === 'production') {
  // (see above)
} else {
  // (3) If no fallback, throw error (non-prod)
  if (!fallback) {
    throw new Error(`${createStringRenderError(message, id)} Error: ${error}`);
  }
  
  console.error(createStringRenderError(message, id), 'Error: ', error);
  // Still renders fallback
}

Behavior:

• 🔴 Throws an error in console (developer sees it clearly)
• 🟡 If there's a fallback, still renders it
• Easier to debug because you get full error details

##Real-World Example: Dynamic Content

// ❌ This violates compiler rules:
const message = "Hello World";
<T>{message}</T>  // Dynamic variable instead of literal

// What happens:

// 1. WITH COMPILER (type: 'babel' or 'swc'):
//    - Build fails immediately
//    - Error during `next build`: "Invalid library usage"
//    - You MUST fix before deployment

// 2. WITH RUNTIME MODE (type: 'none'):
//    - Build succeeds ✅
//    - At runtime, tries to translate dynamically
//    - If translation API is available: translates it
//    - If API fails: falls back to original text
//    - Development: Error logged to console + renders fallback
//    - Production: Warning logged to console + renders fallback
//    - Page still works ✅

##Does It "Throw Errors After Saving a File"?

With Compiler Enabled (Babel/SWC when not using Turbopack):

• ✅ YES - Next.js dev server will catch the error
• Compilation fails immediately
• You see errors in terminal and browser

With Runtime Mode (Turbopack or type: 'none'):

• ❌ NO build errors on save
• File compiles successfully
• Errors only show at runtime when that component renders
• You'll see console warnings/errors when the page loads or interaction occurs
• Page doesn't break, just displays warnings

##Summary Table

##Practical Implications for Turbopack Users

With latest create-next-app (Turbopack enabled):

1. You won't see build errors for unanalyzable patterns
2. You should test thoroughly since errors show only at runtime
3. For static translations, you need explicit patterns:

   // ✅ Good - Static analysis works even in runtime mode
   <T id="welcome">
     <Var>{userName}</Var>, welcome!
   </T>
   
   // ❌ Bad - Runtime mode can't optimize this
   <T>{someVariable}</T>
   

4. Errors are "softer" - warnings logged instead of build failures
5. You get graceful degradation - shows original text instead of breaking

The system is designed to work well in both modes, but the compiler version is stricter (which is safer for production) while runtime mode is more forgiving.

Are there cross-file conflicts that can log errors/warnings? If so, how are they detected and exposed?

Perfect! Now let me compile all this information into a comprehensive answer.

##Cross-File Conflicts: Detection and Exposure

Yes, there are definitely cross-file conflicts that can be detected and logged. They happen at different stages depending on whether you're using the compiler or the CLI. Here's the complete breakdown:

##1. CLI-Level Cross-File Conflicts (When running gt extract or similar)

The CLI scans your entire codebase and detects several types of cross-file issues:

###Type A: Duplicate ID with Mismatched Content

What it detects: If the same id is used in multiple <T> components or dictionary entries across different files but with different content, the CLI will error out.

// file1.tsx
<T id="welcome">Hello World</T>

// file2.tsx - CONFLICT!
<T id="welcome">Hola Mundo</T>  // Same ID, different text!

How it's detected:

// From packages/cli/src/translation/parse.ts
const idHashMap = new Map<string, string>();
const duplicateIds = new Set<string>();

updates = updates.map((update) => {
  if (!update.metadata.id) return update;
  
  const existingHash = idHashMap.get(update.metadata.id);
  if (existingHash) {
    // Same ID found!
    if (existingHash !== update.metadata.hash) {
      // But different content!
      errors.push(
        `Hashes don't match on two components with the same id: ${update.metadata.id}. 
        Check your <T> tags and dictionary entries and make sure you're not accidentally 
        duplicating IDs.`
      );
      duplicateIds.add(update.metadata.id);
    }
  } else {
    idHashMap.set(update.metadata.id, update.metadata.hash);
  }
});

Exposure:

• 🔴 Error in CLI output - shown when running gt extract or gt translate
• Updates with conflicting IDs are filtered out and won't be uploaded
• Build does NOT fail (this is CLI-only, not build-time)

###Type B: Duplicate Content Across Files

What it detects: If the exact same translatable content appears in multiple files, it's deduplicated and merged:

// file1.tsx
<T>Welcome to our app</T>

// file2.tsx
<T>Welcome to our app</T>  // Same content, different file

How it's detected:

// From packages/cli/src/extraction/postProcess.ts
export function dedupeUpdates(updates: Updates): void {
  const mergedByHash = new Map<string, (typeof updates)[number]>();
  
  for (const update of updates) {
    const hash = update.metadata.hash;
    const existing = mergedByHash.get(hash);
    
    if (!existing) {
      mergedByHash.set(hash, update);
      continue;
    }
    
    // Same hash found - merge file paths
    const existingPaths = existing.metadata.filePaths || [];
    const newPaths = update.metadata.filePaths || [];
    
    for (const p of newPaths) {
      if (!existingPaths.includes(p)) {
        existingPaths.push(p);  // ✅ Multiple files associated with one entry
      }
    }
  }
}

Exposure:

• ⚠️ No error - this is intentional behavior
• One entry gets created with multiple file paths attached
• Both files get the same translation

##2. Runtime Dictionary Conflicts (In your app)

###Type A: Duplicate Keys in Dictionary Structure

What it detects: If you have nested dictionaries that flatten to the same key, it throws an error:

// This structure would cause a conflict:
const dictionary = {
  messages: {
    greeting: "Hello",
    "greeting.formal": "Good day"  // Conflicts with nested greeting.formal!
  }
}

How it's detected:

// From packages/react-core/src/internal/flattenDictionary.ts
export default function flattenDictionary(
  dictionary: Dictionary,
  prefix: string = ''
): FlattenedDictionary {
  const flattened: FlattenedDictionary = {};
  
  for (const key in dictionary) {
    const newKey = prefix ? `${prefix}.${key}` : key;
    
    if (flattened.hasOwnProperty(newKey)) {
      throw new Error(`Duplicate key found in dictionary: "${newKey}"`);
    }
    
    flattened[newKey] = dictionary[key];
  }
  
  return flattened;
}

Exposure:

• 🔴 Runtime error thrown when <GTProvider> tries to flatten the dictionary
• Error message: Duplicate key found in dictionary: "..."
• App fails to render

###Type B: GTProvider Prefix ID Mismatch

What it detects: If a <GTProvider> declares a prefix ID but child components use conflicting IDs:

// Parent component
<GTProvider id="admin">
  {/* Child component in different file */}
  <Page>
    <T id="user.welcome">...</T>  // ID doesn't start with "admin" prefix!
  </Page>
</GTProvider>

How it's detected:

// From packages/next/src/errors/createErrors.ts
export const createRequiredPrefixError = (id: string, requiredPrefix: string) =>
  `gt-next Error: <GTProvider> has prefix id "${requiredPrefix}", 
  but a child has id "${id}". Change the <GTProvider> id prop or your 
  dictionary structure to proceed.`;

Exposure:

• 🔴 Runtime error thrown when accessing that translation
• Happens during render time, not at build
• Error clearly shows the mismatch

##3. Build-Time Configuration Conflicts (In next.config.js)

What it detects: If you define GT config in both next.config.js AND gt.config.json with conflicting values:

// next.config.js
export default withGTConfig(nextConfig, {
  locales: ['en', 'es'],
  defaultLocale: 'en'
});

// gt.config.json
{
  "locales": ["en", "fr", "de"],  // ❌ Different locales!
  "defaultLocale": "fr"
}

How it's detected:

// From packages/next/src/config.ts
const conflicts = Object.entries(loadedConfig)
  .filter(([key, value]) => {
    if (!(key in props)) return false;
    
    const propValue = props[key];
    
    // Deep comparison of values
    if (Array.isArray(value)) {
      if (!Array.isArray(propValue)) return true;
      if (value.length !== propValue.length) return true;
      return value.some((v, i) => v !== propValue[i]);
    }
    
    return value !== propValue;
  })
  .map(([key, value]) => 
    `- Key: ${key} Next Config: ${JSON.stringify(props[key])} does not match GT Config: ${JSON.stringify(value)}`
  );

if (conflicts.length) {
  throw new Error(conflictingConfigurationBuildError(conflicts));
}

Exposure:

• 🔴 Build fails immediately with detailed error listing conflicts
• Message shows exactly which keys conflict and what values differ
• Build cannot proceed until resolved

##4. CLI Import Alias Conflicts (When extracting)

What it detects: If your code imports T or Var from another library, the CLI detects and handles it automatically:

// file1.tsx
import { T } from 'some-other-library';  // ❌ Conflict!
import { Var } from 'gt-next';

// The CLI detects this and renames internally to GTT

How it's detected:

// From packages/cli/src/react/jsx/utils/parseAst.ts
generateImportMap(ast, pkg) {
  const importAlias = { TComponent: 'T', VarComponent: 'Var' };
  
  traverse(ast, {
    ImportDeclaration(path) {
      // Check for conflicting imports (not from gt libraries)
      if (source !== pkg) {
        path.node.specifiers.forEach((spec) => {
          if (spec.local.name === 'T') 
            importAlias.TComponent = 'GTT';  // Rename to avoid conflict!
          if (spec.local.name === 'Var') 
            importAlias.VarComponent = 'GTVar';
        });
      }
    }
  });
  
  return { importAlias };
}

Exposure:

• ✅ Automatic resolution - no error, CLI just renames internally
• No warnings unless verbosity is enabled
• Transparent to the user

##Summary Table: Conflict Types and Exposure

##How to Handle Each Conflict

###If you see: "Hashes don't match on two components with the same id"

1. Find both files with the same id
2. Either:
   • Change one of the IDs to be unique
   • Make the content identical
   • Remove the id property to let it auto-generate

###If you see: "Conflicting configuration detected"

1. Remove one of the config sources (either gt.config.json OR next.config.js)
2. Keep only one source of truth
3. Or make them match exactly

###If you see: "Duplicate key found in dictionary"

1. Review your dictionary structure
2. Remove keys that create collisions when flattened
3. Avoid having both greeting and greeting.formal at same level

###If you see: "GTProvider has prefix id but child has different id"

1. Either remove the id prop from <GTProvider>
2. Or ensure all child <T> components use IDs starting with the prefix

##Key Insight: Cross-File Detection Happens in Stages

┌─────────────────────────────────────────────┐
│  1. CLI EXTRACTION (Static Code Analysis)   │
│  ├─ Scans entire codebase                   │
│  ├─ Detects duplicate IDs with conflicts    │
│  ├─ Dedupes identical content               │
│  ├─ Detects import alias conflicts          │
│  └─ Shows errors/warnings in CLI            │
└─────────────────────────────────────────────┘
                      ↓
┌─────────────────────────────────────────────┐
│  2. BUILD TIME (Configuration)              │
│  ├─ Checks next.config.js vs gt.config.json │
│  ├─ Validates compiler options              │
│  └─ Throws build error if mismatched        │
└─────────────────────────────────────────────┘
                      ↓
┌─────────────────────────────────────────────┐
│  3. RUNTIME (App Execution)                 │
│  ├─ Flattens dictionary, detects collisions │
│  ├─ Validates GTProvider prefix matches     │
│  └─ Throws runtime error if conflicts found │
└─────────────────────────────────────────────┘

The compiler itself (Babel/SWC) does NOT detect cross-file conflicts because it processes one file at a time. Cross-file validation happens at the CLI and configuration levels, not during compilation.