Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporters/elasticsearch] remove stale deprecated index option #37094

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .chloggen/elasticsearch-remove-index.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@

# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: breaking

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: exporter/elasticsearch
dmathieu marked this conversation as resolved.
Show resolved Hide resolved

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Remove Index config, which was deprecated since v0.60.0

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [37094]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
dmathieu marked this conversation as resolved.
Show resolved Hide resolved

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []
7 changes: 2 additions & 5 deletions exporter/elasticsearchexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,9 +115,6 @@ Telemetry data will be written to signal specific data streams by default:
logs to `logs-generic-default`, metrics to `metrics-generic-default`, and traces to `traces-generic-default`.
This can be customised through the following settings:

- `index` (DEPRECATED, please use `logs_index` for logs, `metrics_index` for metrics, `traces_index` for traces): The [index] or [data stream] name to publish events to.
The default value is `logs-generic-default`.

- `logs_index`: The [index] or [data stream] name to publish events to. The default value is `logs-generic-default`

- `logs_dynamic_index` (optional): uses resource, scope, or log record attributes to dynamically construct index name.
Expand Down Expand Up @@ -161,7 +158,7 @@ behaviours, which may be configured through the following settings:
- `data_stream.dataset` will always be appended with `.otel`. It is recommended to use with `*_dynamic_index.enabled: true` to route documents to data stream `${data_stream.type}-${data_stream.dataset}-${data_stream.namespace}`.
- Span events are stored in separate documents. They will be routed with `data_stream.type` set to `logs` if `traces_dynamic_index::enabled` is `true`.

- `raw`: Omit the `Attributes.` string prefixed to field names for log and
- `raw`: Omit the `Attributes.` string prefixed to field names for log and
span attributes as well as omit the `Events.` string prefixed to
field names for span events.
- `bodymap`: Provides fine-grained control over the final documents to be ingested.
Expand Down Expand Up @@ -361,4 +358,4 @@ When sending high traffic of metrics to a TSDB metrics data stream, e.g. using O

This will be fixed in a future version of Elasticsearch. A possible workaround would be to use a transform processor to truncate the timestamp, but this will cause duplicate data to be dropped silently.

However, if `@timestamp` precision is not the problem, check your metrics pipeline setup for misconfiguration that causes an actual violation of the [single writer principle](https://opentelemetry.io/docs/specs/otel/metrics/data-model/#single-writer).
However, if `@timestamp` precision is not the problem, check your metrics pipeline setup for misconfiguration that causes an actual violation of the [single writer principle](https://opentelemetry.io/docs/specs/otel/metrics/data-model/#single-writer).
8 changes: 0 additions & 8 deletions exporter/elasticsearchexporter/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,14 +38,6 @@ type Config struct {
// NumWorkers configures the number of workers publishing bulk requests.
NumWorkers int `mapstructure:"num_workers"`

// Index configures the index, index alias, or data stream name events should be indexed in.
//
// https://www.elastic.co/guide/en/elasticsearch/reference/current/indices.html
// https://www.elastic.co/guide/en/elasticsearch/reference/current/data-streams.html
//
// Deprecated: `index` is deprecated and replaced with `logs_index`.
Index string `mapstructure:"index"`

// This setting is required when logging pipelines used.
LogsIndex string `mapstructure:"logs_index"`
// fall back to pure LogsIndex, if 'elasticsearch.index.prefix' or 'elasticsearch.index.suffix' are not found in resource or attribute (prio: resource > attribute)
Expand Down
11 changes: 0 additions & 11 deletions exporter/elasticsearchexporter/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,6 @@ func TestConfig(t *testing.T) {
QueueSize: exporterhelper.NewDefaultQueueConfig().QueueSize,
},
Endpoints: []string{"https://elastic.example.com:9200"},
Index: "",
LogsIndex: "logs-generic-default",
LogsDynamicIndex: DynamicIndexSetting{
Enabled: false,
Expand Down Expand Up @@ -133,7 +132,6 @@ func TestConfig(t *testing.T) {
QueueSize: exporterhelper.NewDefaultQueueConfig().QueueSize,
},
Endpoints: []string{"http://localhost:9200"},
Index: "",
LogsIndex: "my_log_index",
LogsDynamicIndex: DynamicIndexSetting{
Enabled: false,
Expand Down Expand Up @@ -205,7 +203,6 @@ func TestConfig(t *testing.T) {
QueueSize: exporterhelper.NewDefaultQueueConfig().QueueSize,
},
Endpoints: []string{"http://localhost:9200"},
Index: "",
LogsIndex: "logs-generic-default",
LogsDynamicIndex: DynamicIndexSetting{
Enabled: false,
Expand Down Expand Up @@ -284,14 +281,6 @@ func TestConfig(t *testing.T) {
cfg.CloudID = "foo:YmFyLmNsb3VkLmVzLmlvJGFiYzEyMyRkZWY0NTY="
}),
},
{
id: component.NewIDWithName(metadata.Type, "deprecated_index"),
configFile: "config.yaml",
expected: withDefaultConfig(func(cfg *Config) {
cfg.Endpoints = []string{"https://elastic.example.com:9200"}
cfg.Index = "my_log_index"
}),
},
{
id: component.NewIDWithName(metadata.Type, "confighttp_endpoint"),
configFile: "config.yaml",
Expand Down
8 changes: 1 addition & 7 deletions exporter/elasticsearchexporter/factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,6 @@ func createDefaultConfig() component.Config {
return &Config{
QueueSettings: qs,
ClientConfig: httpClientConfig,
Index: "",
LogsIndex: defaultLogsIndex,
LogsDynamicIndex: DynamicIndexSetting{
Enabled: false,
Expand Down Expand Up @@ -111,14 +110,9 @@ func createLogsExporter(
) (exporter.Logs, error) {
cf := cfg.(*Config)

index := cf.LogsIndex
if cf.Index != "" {
set.Logger.Warn("index option are deprecated and replaced with logs_index and traces_index.")
index = cf.Index
}
handleDeprecatedConfig(cf, set.Logger)

exporter := newExporter(cf, set, index, cf.LogsDynamicIndex.Enabled)
exporter := newExporter(cf, set, cf.LogsIndex, cf.LogsDynamicIndex.Enabled)

return exporterhelper.NewLogs(
ctx,
Expand Down
18 changes: 0 additions & 18 deletions exporter/elasticsearchexporter/factory_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,24 +61,6 @@ func TestFactory_CreateTraces(t *testing.T) {
require.NoError(t, exporter.Shutdown(context.Background()))
}

func TestFactory_CreateLogsAndTracesExporterWithDeprecatedIndexOption(t *testing.T) {
factory := NewFactory()
cfg := withDefaultConfig(func(cfg *Config) {
cfg.Endpoints = []string{"http://test:9200"}
cfg.Index = "test_index"
})
params := exportertest.NewNopSettings()
logsExporter, err := factory.CreateLogs(context.Background(), params, cfg)
require.NoError(t, err)
require.NotNil(t, logsExporter)
require.NoError(t, logsExporter.Shutdown(context.Background()))

tracesExporter, err := factory.CreateTraces(context.Background(), params, cfg)
require.NoError(t, err)
require.NotNil(t, tracesExporter)
require.NoError(t, tracesExporter.Shutdown(context.Background()))
}

func TestFactory_DedupDeprecated(t *testing.T) {
factory := NewFactory()
cfg := withDefaultConfig(func(cfg *Config) {
Expand Down
Loading