Added CURSEFORGE_FILES support (#2322)

.github/workflows/build-multiarch.yml

@@ -163,8 +163,7 @@ jobs:
         env:
           MINECRAFT_VERSION: ${{ matrix.mcVersion }}
           VARIANT: ${{ matrix.variant }}
-          MODS_FORGEAPI_KEY: ${{ secrets.MODS_FORGEAPI_KEY }}
+          CF_API_KEY: ${{ secrets.CF_API_KEY }}
-          CF_API_KEY: ${{ secrets.MODS_FORGEAPI_KEY }}
         run: |
           tests/test.sh
 

.github/workflows/verify-pr.yml

@@ -83,8 +83,7 @@ jobs:
         env:
           MINECRAFT_VERSION: ${{ matrix.mcVersion }}
           VARIANT: ${{ matrix.variant }}
-          MODS_FORGEAPI_KEY: ${{ secrets.MODS_FORGEAPI_KEY }}
+          CF_API_KEY: ${{ secrets.CF_API_KEY }}
-          CF_API_KEY: ${{ secrets.MODS_FORGEAPI_KEY }}
           DEBUG: ${{ runner.debug }}
         run: |
           tests/test.sh

Dockerfile

@@ -42,7 +42,7 @@ RUN easy-add --var os=${TARGETOS} --var arch=${TARGETARCH}${TARGETVARIANT} \
   --var version=1.9.0 --var app=mc-server-runner --file {{.app}} \
   --from https://github.com/itzg/{{.app}}/releases/download/{{.version}}/{{.app}}_{{.version}}_{{.os}}_{{.arch}}.tar.gz
 
-ARG MC_HELPER_VERSION=1.32.9
+ARG MC_HELPER_VERSION=1.33.2
 ARG MC_HELPER_BASE_URL=https://github.com/itzg/mc-image-helper/releases/download/${MC_HELPER_VERSION}
 # used for cache busting local copy of mc-image-helper
 ARG MC_HELPER_REV=1

README.md

@@ -895,66 +895,6 @@ You may also download or copy over individual mods using the `MODS` environment
 
   docker run -d -e MODS=https://www.example.com/mods/mod1.jar,/plugins/common,/plugins/special/mod2.jar ...
 
-### ForgeAPI usage to use non-version specific projects
-
-**NOTE:** This potentially could lead to unexpected behavior if the Mod receives an update with unexpected behavior.
-
-This is more complicated because you will be pulling/using the latest mod for the release of your game. To get started make sure you have a [CursedForge API Key](https://docs.curseforge.com/#getting-started). Then use the environmental parameters in your docker build.
-
-Please be aware of the following when using these options for your mods:
-* Mod Release types: Release, Beta, and Alpha.
-* Mod dependencies: Required and Optional
-* Mod family: Fabric, Forge, and Bukkit.
-
-Parameters to use the ForgeAPI:
-
-* `MODS_FORGEAPI_KEY` - Required
-* `MODS_FORGEAPI_FILE` - Required or use MODS_FORGEAPI_PROJECTIDS (Overrides MODS_FORGEAPI_PROJECTIDS)
-* `MODS_FORGEAPI_PROJECTIDS` - Required or use MODS_FORGEAPI_FILE
-* `MODS_FORGEAPI_RELEASES` - Default is release, Options: [Release|Beta|Alpha]
-* `MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES` - Default is False, attempts to download required mods (releaseType Release) defined in Forge.
-* `MODS_FORGEAPI_IGNORE_GAMETYPE` - Default is False, Allows for filtering mods on family type: FORGE, FABRIC, and BUKKIT. (Does not filter for Vanilla or custom)
-* `REMOVE_OLD_FORGEAPI_MODS` - Default is False
-* `REMOVE_OLD_DATAPACKS_DEPTH` - Default is 1
-* `REMOVE_OLD_DATAPACKS_INCLUDE` - Default is *.jar
-
-Example of expected forge api project ids, releases, and key: 
-
-```yaml
-  MODS_FORGEAPI_PROJECTIDS: 306612,256717
-  MODS_FORGEAPI_RELEASES: Release
-  MODS_FORGEAPI_KEY: $WRX...
-```
-
-Example of expected ForgeAPI file format.
-
-**Field Description**: 
-* `name` is currently unused, but can be used to document each entry.
-* `projectId` id is the id found on the CurseForge website for a particular mod
-* `releaseType` Type corresponds to forge's R, B, A icon for each file. Default Release, options are (release|beta|alpha).
-* `fileName` is used for version pinning if latest file will not work for you.
-
-```json
-[
-  {
-      "name": "fabric api",
-      "projectId": "306612",
-      "releaseType": "release"
-  },
-  {
-      "name": "fabric voice mod",
-      "projectId": "416089",
-      "releaseType": "beta"
-  },
-  {
-      "name": "Biomes o plenty",
-      "projectId": "220318",
-      "fileName": "BiomesOPlenty-1.18.1-15.0.0.100-universal.jar",
-      "releaseType": "release"
-  }
-]
-```
-
 ### Generic pack files
 
 To install all the server content (jars, mods, plugins, configs, etc.) from a zip or tgz file, then set `GENERIC_PACK` to the container path or URL of the archive file. This can also be used to apply a CurseForge modpack that is missing a server start script and/or Forge installer.

docs/configuration/jvm-options.md

@@ -70,7 +70,9 @@ To enable remote JMX, such as for profiling with VisualVM or JMC, add the enviro
 
 ## Enable Aikar's Flags
 
-[Aikar has done some research](https://aikar.co/2018/07/02/tuning-the-jvm-g1gc-garbage-collector-flags-for-minecraft/) into finding the optimal JVM flags for GC tuning, which becomes more important as more users are connected concurrently. The set of flags documented there can be added using
+[Aikar has done some research](https://aikar.co/2018/07/02/tuning-the-jvm-g1gc-garbage-collector-flags-for-minecraft/) into finding the optimal JVM flags for GC tuning, which becomes more important as more users are connected concurrently. [PaperMC also has an explanation](https://docs.papermc.io/paper/aikars-flags) of what the JVM flags are doing.
+
+The set of flags documented there can be added using
 
     -e USE_AIKAR_FLAGS=true
 

docs/configuration/misc-options.md

@@ -61,7 +61,9 @@ If you would like to `docker attach` to the Minecraft server console with color
 
 To allow time for players to finish what they're doing during a graceful server shutdown, set `STOP_SERVER_ANNOUNCE_DELAY` to a number of seconds to delay after an announcement is posted by the server.
 
-> **NOTE** be sure to adjust Docker's shutdown timeout accordingly, such as using [the -t option on docker-compose down](https://docs.docker.com/compose/reference/down/).
+!!! warning
+
+    Be sure to adjust Docker's shutdown timeout accordingly, such as using [the -t option on docker-compose down](https://docs.docker.com/compose/reference/down/) or set the [stop_grace_period](https://docs.docker.com/compose/compose-file/05-services/#stop_grace_period) in the compose file.
 
 ## OpenJ9 Specific Options
 

docs/misc/contributing/development.md

@@ -17,7 +17,6 @@ First, build a baseline of the image to include the packages needed by existing
 
 PowerShell: (Example of building and testing ForgeAPI)
 ```powershell
-$env:MODS_FORGEAPI_KEY='$2a$...'
 $env:FOLDER_TO_TEST="forgeapimods_projectids"
 $env:IMAGE_TO_TEST="mc-dev"
 docker build -t $env:IMAGE_TO_TEST .
@@ -36,7 +35,6 @@ docker build --build-arg BASE_IMAGE=$env:BASE_IMAGE -t $env:IMAGE_TO_TEST .
 
 Bash: (Example of building and testing ForgeAPI)
 ```bash
-export MODS_FORGEAPI_KEY='$2a$...'
 export FOLDER_TO_TEST="forgeapimods_file"
 export IMAGE_TO_TEST="mc-dev"
 docker build -t $IMAGE_TO_TEST .

docs/mods-and-plugins/curseforge-files.md

@@ -0,0 +1,45 @@
+# Auto-download from CurseForge
+
+Mods and plugins can be auto-downloaded and upgraded from CurseForge by setting `CURSEFORGE_FILES` to a comma or space delimited list of [project-file references](#project-file-references). References removed from the declaration are automatically cleaned up and setting `CURSEFORGE_FILES` to an empty string removes all previously managed project-files.
+
+A specific file can be omitted from each reference to allow for auto-selecting the newest version of the selected mod/plugin. The resolved `VERSION` and `TYPE` will be taken into consideration for selecting the appropriate file. 
+
+!!! warning "CurseForge API key usage"
+
+    A CurseForge API key must be allocated and set with `CF_API_KEY` [as described here](../types-and-platforms/mod-platforms/auto-curseforge.md#api-key).
+
+## Project-file references
+
+The following formats are supported in the list of project-file references:
+
+- A project page URL, such as `https://www.curseforge.com/minecraft/mc-mods/jei`. _The newest applicable file will be automatically selected._
+- A file page URL, such as `https://www.curseforge.com/minecraft/mc-mods/jei/files/4593548`
+- Project slug, such as `jei`. _The newest applicable file will be automatically selected._
+- Project ID, such as `238222`. _The newest applicable file will be automatically selected._
+- Project slug or ID, `:`, and a file ID, such as `jei:4593548` or `238222:4593548`
+- Project slug or ID, `@`, and a partial filename, such as `jei@10.2.1.1005`
+
+An `@` followed by the container path to a listing file can also be provided as a project-file reference. Each line is processed as a reference where blank lines and comments that start with `#` are ignored.
+
+For example, `CURSEFORGE_FILES` can be set to "@/extras/cf-mods.txt" where the container file `/extras/cf-mods.txt` contains
+
+```text
+# This comment is ignored
+jei:10.2.1.1005
+
+# This and previous blank line are ignore
+geckolib
+aquaculture
+naturalist
+```
+
+!!! tip "Docker Compose"
+
+    Making use of the space delimited option, compose file declarations can be organized nicely with a [multi-line string](https://yaml-multiline.info/), such as
+    
+    ```yaml
+          CURSEFORGE_FILES: |
+            geckolib
+            aquaculture
+            naturalist
+    ```

docs/mods-and-plugins/forge-api.md

@@ -1,65 +0,0 @@
-# Forge API
-
-## ForgeAPI usage to use non-version specific projects
-
-!!! warning "Deprecated"
-    This approach will soon be deprecated in favor of a variation of `AUTO_CURSEFORGE`.
-
-!!! warning
-    This potentially could lead to unexpected behavior if the Mod receives an update with unexpected behavior.
-
-This is more complicated because you will be pulling/using the latest mod for the release of your game. To get started make sure you have a [CursedForge API Key](https://docs.curseforge.com/#getting-started). Then use the environmental parameters in your docker build.
-
-Please be aware of the following when using these options for your mods:
-* Mod Release types: Release, Beta, and Alpha.
-* Mod dependencies: Required and Optional
-* Mod family: Fabric, Forge, and Bukkit.
-
-Parameters to use the ForgeAPI:
-
-* `MODS_FORGEAPI_KEY` - Required
-* `MODS_FORGEAPI_FILE` - Required or use MODS_FORGEAPI_PROJECTIDS (Overrides MODS_FORGEAPI_PROJECTIDS)
-* `MODS_FORGEAPI_PROJECTIDS` - Required or use MODS_FORGEAPI_FILE
-* `MODS_FORGEAPI_RELEASES` - Default is release, Options: [Release|Beta|Alpha]
-* `MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES` - Default is False, attempts to download required mods (releaseType Release) defined in Forge.
-* `MODS_FORGEAPI_IGNORE_GAMETYPE` - Default is False, Allows for filtering mods on family type: FORGE, FABRIC, and BUKKIT. (Does not filter for Vanilla or custom)
-* `REMOVE_OLD_FORGEAPI_MODS` - Default is False
-* `REMOVE_OLD_DATAPACKS_DEPTH` - Default is 1
-* `REMOVE_OLD_DATAPACKS_INCLUDE` - Default is *.jar
-
-Example of expected forge api project ids, releases, and key:
-
-```yaml
-  MODS_FORGEAPI_PROJECTIDS: 306612,256717
-  MODS_FORGEAPI_RELEASES: Release
-  MODS_FORGEAPI_KEY: $WRX...
-```
-
-Example of expected ForgeAPI file format.
-
-**Field Description**:
-* `name` is currently unused, but can be used to document each entry.
-* `projectId` id is the id found on the CurseForge website for a particular mod
-* `releaseType` Type corresponds to forge's R, B, A icon for each file. Default Release, options are (release|beta|alpha).
-* `fileName` is used for version pinning if latest file will not work for you.
-
-```json
-[
-  {
-      "name": "fabric api",
-      "projectId": "306612",
-      "releaseType": "release"
-  },
-  {
-      "name": "fabric voice mod",
-      "projectId": "416089",
-      "releaseType": "beta"
-  },
-  {
-      "name": "Biomes o plenty",
-      "projectId": "220318",
-      "fileName": "BiomesOPlenty-1.18.1-15.0.0.100-universal.jar",
-      "releaseType": "release"
-  }
-]
-```

examples/curseforge-files/docker-compose.yml

@@ -0,0 +1,17 @@
+version: "3.8"
+
+services:
+  mc:
+    image: itzg/minecraft-server
+    environment:
+      EULA: true
+      TYPE: FORGE
+      # allocate from https://console.curseforge.com/ and set in .env file
+      CF_API_KEY: ${CF_API_KEY}
+      VERSION: 1.19.2
+      CURSEFORGE_FILES: |
+        geckolib
+        aquaculture
+        naturalist
+    ports:
+      - "25565:25565"

examples/docker-compose-rconcmd.yml

@@ -10,10 +10,12 @@ services:
     environment:
       EULA: "TRUE"
       TYPE: FABRIC
-      MODS_FORGEAPI_PROJECTIDS: 433175
+      MEMORY: "2G"
-      MODS_FORGEAPI_KEY: ${MODS_FORGEAPI_KEY}
+      CURSEFORGE_FILES: |
-      MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES: "TRUE"
+        fabric-api
-      REMOVE_OLD_FORGEAPI_MODS: "TRUE"
+        chunky-pregenerator
+      # allocate from https://console.curseforge.com/ and set in .env file
+      CF_API_KEY: ${CF_API_KEY}
       # YAML Heredoc, be sure to use '|-' this will remove the first newline and final new line.
       # This is versus '|' that will leaving with two empty strings at top and bottom.
       RCON_CMDS_STARTUP:  |-

scripts/start-setupDatapack

@@ -84,4 +84,4 @@ if [[ ${VANILLATWEAKS_FILE} || ${VANILLATWEAKS_SHARECODE} ]]; then
     --pack-files="$VANILLATWEAKS_FILE"
 fi
 
-exec "${SCRIPTS:-/}start-setupForgeApiMods" "$@"
+exec "${SCRIPTS:-/}start-setupModpack" "$@"

scripts/start-setupForgeApiMods

@@ -1,244 +0,0 @@
-#!/bin/bash
-
-set -e -o pipefail
-
-: "${MODS_FORGEAPI_KEY:=}"
-: "${REMOVE_OLD_FORGEAPI_MODS:=false}"
-: "${MODS_FORGEAPI_PROJECTIDS:=}"
-: "${MODS_FORGEAPI_FILE:=}"
-: "${MODS_FORGEAPI_RELEASES:=RELEASE}"
-: "${MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES:=false}"
-: "${MODS_FORGEAPI_IGNORE_GAMETYPE:=false}"
-: "${REMOVE_OLD_MODS_DEPTH:=1} "
-: "${REMOVE_OLD_MODS_INCLUDE:=*.jar,*-version.json}"
-
-#  FORGEAPI_BASE_URL used in manifest downloads below
-FORGEAPI_BASE_URL=${FORGEAPI_BASE_URL:-https://api.curseforge.com/v1}
-RELEASE_NUMBER_FILTER=1
-MINECRAFT_GAME_ID=432
-FILTER_BY_FAMILY=false
-DOWNLOADED_MODIDS=()
-out_dir=/data/mods
-
-# shellcheck source=start-utils
-. "${SCRIPTS:-/}start-utils"
-isDebugging && set -x
-
-# Remove old mods/plugins
-if isTrue "${REMOVE_OLD_FORGEAPI_MODS}"; then
-  removeOldMods "/data/mods"
-fi
-
-# Family filter is on by default for Forge, Fabric, and Bukkit
-updateFamilyFilter(){
-  if isFamily "FORGE" "FABRIC" "BUKKIT"; then
-    FILTER_BY_FAMILY=true
-  fi
-}
-
-ensureModKey(){
-  if [ -z "$MODS_FORGEAPI_KEY" ]; then
-    log "ERROR: MODS_FORGEAPI_KEY REQUIRED to Connect to FORGE API, you supplied: ${MODS_FORGEAPI_KEY}"
-    exit 2
-  fi
-}
-
-# Set the global release type per the text.
-# NOTE: downcasing release type for comparing types.
-updateReleaseNumber(){
-  releaseType=$1
-  if [ "release" = "${releaseType,,}" ] || [ 1 = "${releaseType,,}" ]; then
-    RELEASE_NUMBER_FILTER=1
-  elif [ "beta" = "${releaseType,,}" ] || [ 2 = "${releaseType,,}" ]; then
-    RELEASE_NUMBER_FILTER=2
-  elif [ "alpha" = "${releaseType,,}" ] || [ 3 = "${releaseType,,}" ]; then
-    RELEASE_NUMBER_FILTER=3
-  fi
-}
-
-retrieveVersionTypeNumber(){
-  VERSION_NAME=$(get_major_version "$VERSION")
-  minecraft_types=$(curl -X GET -s \
-    "${FORGEAPI_BASE_URL}/games/${MINECRAFT_GAME_ID}/version-types" \
-    -H 'Accept: application/json' -H 'x-api-key: '${MODS_FORGEAPI_KEY}'')
-
-  if [ ! "$minecraft_types" ]; then
-    log "ERROR: unable to retrieve version types for ${VERSION_NAME} from ForgeAPI. Check Forge API key or supplied Minecraft version"
-    exit 2
-  fi
-
-  TYPE_ID=$(jq -n "$minecraft_types" | jq --arg VERSION_NAME "$VERSION_NAME" -jc '
-      .data[]? | select(.name==$VERSION_NAME) | .id')
-
-  if [ ! "$TYPE_ID" ]; then
-    log "ERROR: unable to retrieve version types for ${VERSION_NAME} from ForgeAPI"
-    exit 2
-  fi
-}
-
-modFileByProjectID(){
-  project_id=$(echo "$1" | tr -d '"')
-  project_id_release_type=$2
-  project_id_file_name=$3
-  unset PROJECT_FILE
-
-  # if Type id isn't defined use minecraft version to go get it.
-  if [ ! "$TYPE_ID" ]; then
-    retrieveVersionTypeNumber
-  fi
-
-  # JQ is struggling with larger page sizes so having to pagination for mods with a lot of releases
-  pageSize=42
-  index=0
-  total_count=1
-
-  while [ $index -lt $total_count ]; do
-    project_files=$(curl -X GET -s \
-      "${FORGEAPI_BASE_URL}/mods/${project_id}/files?gameVersionTypeId=${TYPE_ID}&index=${index}&pageSize=${pageSize}" \
-      -H 'Accept: application/json' -H 'x-api-key: '${MODS_FORGEAPI_KEY}'')
-
-    if [ ! "$project_files" ]; then
-      log "ERROR: unable to retrieve any project id files for ${project_id} from ForgeAPI"
-      exit 2
-    fi
-    # Use project files to grab out the total count of mods.
-    total_count=$(jq -n "$project_files" | jq -c '.pagination.totalCount' )
-
-    # Checking for a individual release type input, if not use global
-    if [ "$project_id_release_type" ]; then
-      updateReleaseNumber "$project_id_release_type"
-      unset project_id_release_type 
-    else
-      updateReleaseNumber $MODS_FORGEAPI_RELEASES
-    fi
-
-    # grabs the highest ID of the releaseTypes selected.
-    # Default is 1 for Release, Beta is 2, and Alpha is 3. Using less than we can validate highest release.
-    if [ "$project_id_file_name" ]; then
-      # Looks for file by name
-      current_project_file=$(jq -n "$project_files" | jq --arg FILE_NAME "$project_id_file_name" -jc '
-        .data | map(select(.fileName<=($FILE_NAME))) | .[0] // empty')
-    elif isFalse "${MODS_FORGEAPI_IGNORE_GAMETYPE}" && $FILTER_BY_FAMILY ; then
-      # Looks for file by version and server type in lowercase
-      current_project_file=$(jq -n "$project_files" | jq --arg RELEASE_FILTER "$RELEASE_NUMBER_FILTER" --arg GAME_TYPE "${FAMILY,,}" --arg VERSION "$VANILLA_VERSION" -jc '
-        .data | sort_by(.id) | reverse | map(select(.gameVersions[] | ascii_downcase | contains ($GAME_TYPE))) |  map(select(.gameVersions[] | contains ($VERSION))) | map(select(.releaseType<=($RELEASE_FILTER|tonumber))) | .[0] // empty')
-    else
-      # Looks for file by version only.
-      current_project_file=$(jq -n "$project_files" | jq --arg RELEASE_FILTER "$RELEASE_NUMBER_FILTER" --arg VERSION "$VANILLA_VERSION" -jc '
-        .data | sort_by(.id) | reverse | map(select(.gameVersions[] | contains ($VERSION))) | map(select(.releaseType<=($RELEASE_FILTER|tonumber))) | .[0] // empty')
-    fi
-
-    # Logic to grab the latest release over the entire pagination
-    if [ ! "$PROJECT_FILE" ]; then
-      PROJECT_FILE=$current_project_file
-    elif [ "$current_project_file" ]; then
-      current_project_file_id=$(jq -n "$current_project_file" | jq -jc '.id // empty' )
-      PROJECT_FILE_ID=$(jq -n "$PROJECT_FILE" | jq -jc '.id // empty' )
-      if (( current_project_file_id > PROJECT_FILE_ID )); then
-        PROJECT_FILE=$current_project_file
-      fi
-    fi
-    
-    # check to see if we have gone to far or lost our index and exit with an error
-    if [ -z "$index" ] || [ -z "$total_count" ] || [ $index -ge "$total_count" ]; then
-      log "ERROR: Unable to retrieve any files for ${project_id} from ForgeAPI also Validate files have release type associated with no. ${RELEASE_NUMBER_FILTER}"
-      exit 2
-    fi
-    # Increment start index to new set.
-    index=$((index + pageSize))
-  done
-  if [ ! "$PROJECT_FILE" ]; then
-    log "ERROR: Unable to retrieve any files for ${project_id}, Release Type: ${RELEASE_NUMBER_FILTER}, FAMILY_TYPE: ${FAMILY,,}"
-    exit 2
-  fi
-}
-
-downloadModPackfromModFile() {
-  if [ ! "$PROJECT_FILE" ]; then
-    log "ERROR: Project File not found from the ForgeAPI"
-    exit 2
-  fi
-  # trys to make the output directory incase it doesnt exist.
-  mkdir -p "$out_dir"
-  debug "DEBUG: PROJECT_FILE: ${PROJECT_FILE}"
-  # grabs needed values from our json return
-  file_name=$(jq -n "$PROJECT_FILE" | jq -jc '.fileName // empty' )
-  download_url=$(jq -n "$PROJECT_FILE" | jq -jc '.downloadUrl // empty' )
-  mod_id=$(jq -n "$PROJECT_FILE" | jq -jc '.modId // empty' )
-
-  if [ ! -f "${out_dir}/${file_name}" ]; then
-    echo "Downloading ${download_url}"
-    # Track the mods we have downloaded.
-    DOWNLOADED_MODIDS+=("${mod_id}")
-    if ! get --skip-up-to-date -o "${out_dir}/${file_name}" "${download_url}"; then
-      log "ERROR: failed to download from ${download_url}"
-      exit 2
-    fi
-  fi
-}
-
-downloadDependencies(){
-  if [ "$PROJECT_FILE" ]; then
-    dependencies=$(jq -n "$PROJECT_FILE" | jq -jc '.dependencies' )
-    required_dependencies=$(jq -n "$dependencies" | jq --arg REQUIRED_FILTER "3" -jc '
-      map(select(.relationType==($REQUIRED_FILTER|tonumber)))')
-    if [ "$required_dependencies" ]; then
-      while read -r current_dependency; do
-        mod_id=$(jq -n "$current_dependency" | jq -jc '.modId' )
-        # Validate we have not tried to download the mod yet.
-        if [[ ! "${DOWNLOADED_MODIDS[*]}" =~ $mod_id ]]; then
-          modFileByProjectID "$mod_id" "release"
-          downloadModPackfromModFile
-        fi
-      # needs to be piped in to keep look in main process
-      done < <(jq -n "$required_dependencies" | jq -c '.[]?')
-    fi
-  fi
-}
-
-# Use forge api json file to filter and download the correct mods
-if [ "$MODS_FORGEAPI_FILE" ] && [ -z "$MODS_FORGEAPI_PROJECTIDS" ]; then
-  ensureModKey
-  updateFamilyFilter
-  if [ ! -f "$MODS_FORGEAPI_FILE" ]; then
-    log "ERROR: given MODS_FORGEAPI_FILE file does not exist"
-    exit 2
-  fi
-  debug "DEBUG: MODS_FORGEAPI_KEY: ${MODS_FORGEAPI_FILE}"
-  # Needs loop here to look up release types befor calling download.
-  while read -r current_project; do
-    debug "DEBUG: current_project: ${current_project}"
-    # Per stack overflow we can use //empty to return empty string that works with -z
-    project_id=$(jq -n "$current_project" | jq -r '.projectId // empty' )
-    current_release_type=$(jq -n "$current_project" | jq -r '.releaseType // empty' )
-    current_file_name=$(jq -n "$current_project" | jq -r '.fileName // empty' )
-
-    # Validate we have not tried to download the mod yet.
-    if [[ ! "${DOWNLOADED_MODIDS[*]}" =~ $project_id ]]; then
-      modFileByProjectID "$project_id" "$current_release_type" "$current_file_name"
-      downloadModPackfromModFile
-      if isTrue "${MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES}"; then
-        downloadDependencies
-      fi
-    fi
-  # needs to be piped in to keep look in main process
-  done < <(jq -c '.[]?' $MODS_FORGEAPI_FILE)
-fi
-
-# Use only project ids and global release data.
-if [ "$MODS_FORGEAPI_PROJECTIDS" ] && [ -z "$MODS_FORGEAPI_FILE" ]; then
-  ensureModKey
-  updateFamilyFilter
-  for project_id in ${MODS_FORGEAPI_PROJECTIDS//,/ }; do
-    # Validate we have not tried to download the mod yet.
-    if [[ ! "${DOWNLOADED_MODIDS[*]}" =~ $project_id ]]; then
-      modFileByProjectID $project_id 
-      downloadModPackfromModFile
-      if isTrue "${MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES}"; then
-        downloadDependencies
-      fi
-    fi
-  done
-fi
-
-exec "${SCRIPTS:-/}start-setupModpack" "$@"

scripts/start-setupModpack

@@ -143,51 +143,6 @@ function handleListings() {
   fi
 }
 
-function handleCurseForgeManifest() {
-if [[ "$MANIFEST" ]]; then
-    if [[ -e "$MANIFEST" ]]; then
-        EFFECTIVE_MANIFEST_FILE=$MANIFEST
-    elif isURL "$MANIFEST"; then
-        EFFECTIVE_MANIFEST_FILE=/tmp/manifest.json
-        EFFECTIVE_MANIFEST_URL=$(curl -Ls -o /dev/null -w "%{effective_url}" "$MANIFEST")
-        curl -Ls -o $EFFECTIVE_MANIFEST_FILE "$EFFECTIVE_MANIFEST_URL"
-    else
-        log "MANIFEST='$MANIFEST' is not a valid manifest url or location"
-        exit 2
-    fi
-
-case "X$EFFECTIVE_MANIFEST_FILE" in
-  X*.json)
-    if [ -f "${EFFECTIVE_MANIFEST_FILE}" ]; then
-      MOD_DIR=${FTB_BASE_DIR:-/data}/mods
-      if [ ! -d "$MOD_DIR" ]
-      then
-        log "Creating mods dir $MOD_DIR"
-        mkdir -p "$MOD_DIR"
-      fi
-      log "Starting manifest download..."
-      jq -r '.files[] | (.projectID|tostring) + " " + (.fileID|tostring)' "${EFFECTIVE_MANIFEST_FILE}" | while read -r p f
-      do
-        if [ ! -f $MOD_DIR/${p}_${f}.jar ]
-        then
-          redirect_url="$(curl -Ls -o /dev/null -w "%{effective_url}" "${CURSE_URL_BASE}/${p}")"
-          url="$redirect_url/download/${f}/file"
-          log Downloading curseforge mod $url
-          #  Manifest usually doesn't have mod names. Using id should be fine, tho
-          curl -sSL "${url}" -o $MOD_DIR/${p}_${f}.jar
-        fi
-      done
-    else
-      log "Could not find manifest file, insufficient privileges, or malformed path."
-    fi
-    ;;
-  *)
-    log "Invalid manifest file for modpack. Please make sure it is a .json file."
-    ;;
-esac
-fi
-}
-
 function handleGenericPacks() {
   : "${GENERIC_PACKS:=${GENERIC_PACK}}"
   : "${GENERIC_PACKS_PREFIX:=}"
@@ -296,16 +251,51 @@ function handleModrinthProjects() {
   fi
 }
 
+function handleCurseForgeFiles() {
+  args=()
+  if usesMods && ! usesPlugins; then
+    args+=(--default-category mc-mods)
+  elif usesPlugins && ! usesMods; then
+    args+=(--default-category bukkit-plugins)
+  fi
+
+  case "${TYPE,,}" in
+    forge|fabric|quilt)
+      args+=(--mod-loader "$TYPE")
+      ;;
+  esac
+
+  # shellcheck disable=SC2086
+  # since we want CURSEFORGE_FILES to expand
+  mc-image-helper curseforge-files \
+    "${args[@]}" \
+    ${CURSEFORGE_FILES}
+}
+
 handlePackwiz
 
 handleModpackZip
 
 handleListings
 
-handleCurseForgeManifest
+if [[ $MANIFEST ]]; then
+  log "ERROR: MANIFEST is no longer supported."
+  log "       Use MOD_PLATFORM=AUTO_CURSEFORGE and CF_MODPACK_MANIFEST instead"
+  exit 1
+fi
+
+if [[ $MODS_FORGEAPI_KEY || $MODS_FORGEAPI_FILE || $MODS_FORGEAPI_PROJECTIDS ]]; then
+  log "ERROR the MODS_FORGEAPI_FILE / MODS_FORGEAPI_PROJECTIDS feature is no longer supported"
+  log "      Use CURSEFORGE_FILES instead."
+  exit 1
+fi
 
 handleGenericPacks
 
 handleModrinthProjects
 
+if usesMods || usesPlugins; then
+  handleCurseForgeFiles
+fi
+
 exec "${SCRIPTS:-/}start-setupModconfig" "$@"

tests/setuponlytests/cf_files/docker-compose.yml

@@ -0,0 +1,22 @@
+version: "3"
+
+services:
+  mc:
+    restart: "no"
+    image: ${IMAGE_TO_TEST:-itzg/minecraft-server}
+    environment:
+      EULA: "TRUE"
+      SETUP_ONLY: "TRUE"
+      TYPE: FABRIC
+      VERSION: 1.18.2
+      # Contains mix of Forge and Fabric mods
+      CURSEFORGE_FILES: |
+        https://www.curseforge.com/minecraft/mc-mods/clumps/files/4153343
+        jei
+        306612
+        @/extras/listing.txt
+      CF_API_KEY: ${CF_API_KEY}
+    volumes:
+      - ./data:/data
+      - ./fake.jar:/servers/fake.jar:ro
+      - ./listing.txt:/extras/listing.txt:ro

tests/setuponlytests/cf_files/listing.txt

@@ -0,0 +1 @@
+architectury-api@4.11.93

tests/setuponlytests/cf_files/require.sh

@@ -0,0 +1 @@
+[[ $CF_API_KEY ]] || exit 1

tests/setuponlytests/cf_files/verify.sh

@@ -0,0 +1,5 @@
+mc-image-helper assert fileExists "/data/mods/architectury-*-fabric.jar"
+mc-image-helper assert fileExists "/data/mods/Clumps-fabric-*.jar"
+mc-image-helper assert fileExists "/data/mods/fabric-api-*.jar"
+mc-image-helper assert fileExists "/data/mods/jei-*-fabric-*.jar"
+

tests/setuponlytests/forgeapimods_file/docker-compose.yml

@@ -1,27 +0,0 @@
-version: "3"
-
-services:
-  mc:
-    restart: "no"
-    image: ${IMAGE_TO_TEST:-itzg/minecraft-server}
-    environment:
-      EULA: "TRUE"
-      SETUP_ONLY: "TRUE"
-      DEBUG: "FALSE"
-      # Using custom to bypass Fabric setup
-      TYPE: CUSTOM
-      # Using family to test FORGEAPI Family filter.
-      FAMILY: FABRIC
-      CUSTOM_SERVER: /servers/fake.jar
-      VERSION: 1.18.2
-      MODS_FORGEAPI_FILE: /config/forgeapi_mods.json
-      # Key is passed by Github Workflow
-      # This should be coming from github secrets.
-      MODS_FORGEAPI_KEY: ${MODS_FORGEAPI_KEY}
-      REMOVE_OLD_FORGEAPI_MODS: "TRUE"
-      # Validates that Fabric API gets download as a dependency.
-      MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES: "TRUE"
-    volumes:
-      - ./forgeapi_mods.json:/config/forgeapi_mods.json:ro
-      - ./data:/data
-      - ./fake.jar:/servers/fake.jar

tests/setuponlytests/forgeapimods_file/forgeapi_mods.json

@@ -1,11 +0,0 @@
-[
-  {
-    "name": "Flan (Land Claim Tool)",
-    "projectId": "404578"
-  },
-  {
-    "name": "Fabric Voice Mod",
-    "projectId": "416089",
-    "releaseType": "beta"
-  }
-]

tests/setuponlytests/forgeapimods_file/require.sh

@@ -1,2 +0,0 @@
-[[ $MINECRAFT_VERSION == LATEST ]] || exit 1
-[[ $MODS_FORGEAPI_KEY ]] || exit 1

tests/setuponlytests/forgeapimods_file/verify.sh

@@ -1,5 +0,0 @@
-# Validates specific beta call out for specific mod:
-mc-image-helper assert fileExists "/data/mods/voicechat-fabric*"
-mc-image-helper assert fileExists "/data/mods/flan*"
-# Dependent of flan:
-mc-image-helper assert fileExists "/data/mods/fabric-api*"

tests/setuponlytests/forgeapimods_gametype/docker-compose.yml

@@ -1,30 +0,0 @@
-version: "3"
-
-services:
-  mc:
-    restart: "no"
-    image: ${IMAGE_TO_TEST:-itzg/minecraft-server}
-    environment:
-      EULA: "TRUE"
-      SETUP_ONLY: "TRUE"
-      DEBUG: "FALSE"
-      # Using custom to bypass Fabric setup
-      TYPE: CUSTOM
-      # Using family to test FORGEAPI Family filter.
-      FAMILY: FABRIC
-      CUSTOM_SERVER: /servers/fake.jar
-      VERSION: 1.18.2
-      # Validate Skip Gametype Filter:
-      MODS_FORGEAPI_IGNORE_GAMETYPE: "TRUE"
-      # Validates that Biomes does not download terrablender
-      # Using default false for testing: 
-      MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES: "FALSE"
-      # Contains mix of Forge and Fabric mods
-      MODS_FORGEAPI_PROJECTIDS: 416089,493246
-      # Allows for Beta releases of 416089 the Fabric Voice Mod 
-      MODS_FORGEAPI_RELEASES: BETA
-      MODS_FORGEAPI_KEY: ${MODS_FORGEAPI_KEY}
-      REMOVE_OLD_FORGEAPI_MODS: "TRUE"
-    volumes:
-      - ./data:/data
-      - ./fake.jar:/servers/fake.jar

tests/setuponlytests/forgeapimods_gametype/require.sh

@@ -1,2 +0,0 @@
-[[ $MINECRAFT_VERSION == LATEST ]] || exit 1
-[[ $MODS_FORGEAPI_KEY ]] || exit 1

tests/setuponlytests/forgeapimods_gametype/verify.sh

@@ -1,5 +0,0 @@
-# testing dependencies don't get downloaded when download dependencies is set to false.
-mc-image-helper assert fileExists "/data/mods/voicechat-fabric*"
-mc-image-helper assert fileExists "/data/mods/flan*"
-# Dependent of flan, but dependencies are set to false:
-! mc-image-helper assert fileExists "/data/mods/fabric-api*"

tests/setuponlytests/forgeapimods_projectids/docker-compose.yml

@@ -1,26 +0,0 @@
-version: "3"
-
-services:
-  mc:
-    restart: "no"
-    image: ${IMAGE_TO_TEST:-itzg/minecraft-server}
-    environment:
-      EULA: "TRUE"
-      SETUP_ONLY: "TRUE"
-      # Using custom to bypass Fabric setup
-      TYPE: CUSTOM
-      # Validate Skip Gametype Filter for vanilla 
-      #  - Currently we do not support filtering on vanilla.
-      FAMILY: VANILLA
-      CUSTOM_SERVER: /servers/fake.jar
-      VERSION: 1.18.2
-      MODS_FORGEAPI_DOWNLOAD_DEPENDENCIES: "TRUE"
-      # Contains mix of Forge and Fabric mods
-      MODS_FORGEAPI_PROJECTIDS: 416089,419697
-      # Allows for Beta releases of 416089 the Fabric Voice Mod 
-      MODS_FORGEAPI_RELEASES: BETA
-      MODS_FORGEAPI_KEY: ${MODS_FORGEAPI_KEY}
-      REMOVE_OLD_FORGEAPI_MODS: "TRUE"
-    volumes:
-      - ./data:/data
-      - ./fake.jar:/servers/fake.jar

tests/setuponlytests/forgeapimods_projectids/require.sh

@@ -1,2 +0,0 @@
-[[ $MINECRAFT_VERSION == LATEST ]] || exit 1
-[[ $MODS_FORGEAPI_KEY ]] || exit 1

tests/setuponlytests/forgeapimods_projectids/verify.sh

@@ -1,6 +0,0 @@
-# No family filter applied, DO NOT use Fabric or Forge specific name validation as it may cause random breakage.
-mc-image-helper assert fileExists "/data/mods/voicechat-fabric-1*"
-# Should be pull v4 and higher for 1.18.2:
-mc-image-helper assert fileExists "/data/mods/architectury-4*"
-mc-image-helper assert fileExists "/data/mods/fabric-api*"
-